{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请在Capital Bikeshare（美国Washington, D.C.的一个共享单车公司）提供的自行车数据上进行回归分析。训练数据为2011年的数据，要求预测2012年每天的单车共享数量。\n",
    "\n",
    "需要提交代码文件，并出必要的结果解释。\n",
    "\n",
    "1)训练数据和测试数据分割（请将2012年的数据作为测数据）；（20分）\n",
    "\n",
    "2)适当的特征工程（及数据探索）;（20分）\n",
    "\n",
    "提示：\n",
    "a)有些特征看起是数据值特征，其实是类别型特征，如月份、季节\n",
    "b)数值型特征归一化\n",
    "c)可以弃一些不必要的特征\n",
    "\n",
    "3)岭回归，并选择最佳的正则参数；（30分）\n",
    "   a)参数调优\n",
    "   b)结果可视化\n",
    "4)Lasso，并选择最佳的正则参数；（30分）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1、导入必要工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np # 矩阵操作\n",
    "import pandas as pd # SQL数据处理\n",
    "\n",
    "from sklearn.metrics import r2_score # 评价回归预测模型的性能\n",
    "\n",
    "import matplotlib.pyplot as plt # 画图 可视化\n",
    "import seaborn as sns # 数据可视化工具包\n",
    "\n",
    "# 图形出现在Notebook里而不是新窗口\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2、数据探索\n",
    "2.1、读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>dteday</th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weekday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>casual</th>\n",
       "      <th>registered</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.344167</td>\n",
       "      <td>0.363625</td>\n",
       "      <td>0.805833</td>\n",
       "      <td>0.160446</td>\n",
       "      <td>331</td>\n",
       "      <td>654</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2011-01-02</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.363478</td>\n",
       "      <td>0.353739</td>\n",
       "      <td>0.696087</td>\n",
       "      <td>0.248539</td>\n",
       "      <td>131</td>\n",
       "      <td>670</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2011-01-03</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.196364</td>\n",
       "      <td>0.189405</td>\n",
       "      <td>0.437273</td>\n",
       "      <td>0.248309</td>\n",
       "      <td>120</td>\n",
       "      <td>1229</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>2011-01-04</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>0.212122</td>\n",
       "      <td>0.590435</td>\n",
       "      <td>0.160296</td>\n",
       "      <td>108</td>\n",
       "      <td>1454</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>2011-01-05</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.226957</td>\n",
       "      <td>0.229270</td>\n",
       "      <td>0.436957</td>\n",
       "      <td>0.186900</td>\n",
       "      <td>82</td>\n",
       "      <td>1518</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant      dteday  season  yr  mnth  holiday  weekday  workingday  \\\n",
       "0        1  2011-01-01       1   0     1        0        6           0   \n",
       "1        2  2011-01-02       1   0     1        0        0           0   \n",
       "2        3  2011-01-03       1   0     1        0        1           1   \n",
       "3        4  2011-01-04       1   0     1        0        2           1   \n",
       "4        5  2011-01-05       1   0     1        0        3           1   \n",
       "\n",
       "   weathersit      temp     atemp       hum  windspeed  casual  registered  \\\n",
       "0           2  0.344167  0.363625  0.805833   0.160446     331         654   \n",
       "1           2  0.363478  0.353739  0.696087   0.248539     131         670   \n",
       "2           1  0.196364  0.189405  0.437273   0.248309     120        1229   \n",
       "3           1  0.200000  0.212122  0.590435   0.160296     108        1454   \n",
       "4           1  0.226957  0.229270  0.436957   0.186900      82        1518   \n",
       "\n",
       "    cnt  \n",
       "0   985  \n",
       "1   801  \n",
       "2  1349  \n",
       "3  1562  \n",
       "4  1600  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# set path to where the data of bike-sharing-set\n",
    "dpath = 'D:/AI/Bike-Sharing-Dataset/'\n",
    "data = pd.read_csv(dpath + \"day.csv\")\n",
    "\n",
    "# 通过观察前5行，了解数据每列（特征）的情况\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.2、数据基本信息\n",
    "样本数目、特征维数，每个特征的类型、空值样本的数目、数据类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(731, 16)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.3、数据探索\n",
    "对数据的探索有助于我们在第三步中根据数据的特点选择合适的模型类型\n",
    "\n",
    "2.4、数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season</th>\n",
       "      <th>yr</th>\n",
       "      <th>mnth</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>weathersit</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.363625</td>\n",
       "      <td>0.805833</td>\n",
       "      <td>0.160446</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.353739</td>\n",
       "      <td>0.696087</td>\n",
       "      <td>0.248539</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.189405</td>\n",
       "      <td>0.437273</td>\n",
       "      <td>0.248309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.212122</td>\n",
       "      <td>0.590435</td>\n",
       "      <td>0.160296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.229270</td>\n",
       "      <td>0.436957</td>\n",
       "      <td>0.186900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   season  yr  mnth  holiday  workingday  weathersit     atemp       hum  \\\n",
       "0       1   0     1        0           0           2  0.363625  0.805833   \n",
       "1       1   0     1        0           0           2  0.353739  0.696087   \n",
       "2       1   0     1        0           1           1  0.189405  0.437273   \n",
       "3       1   0     1        0           1           1  0.212122  0.590435   \n",
       "4       1   0     1        0           1           1  0.229270  0.436957   \n",
       "\n",
       "   windspeed  \n",
       "0   0.160446  \n",
       "1   0.248539  \n",
       "2   0.248309  \n",
       "3   0.160296  \n",
       "4   0.186900  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 从原始数据中分离输入特征X和输出y\n",
    "y = data['cnt'].values # 根据作业说明文档，casual、registered、cnt都是输出y，但作业只需对cnt进行预测\n",
    "\n",
    "X = data.drop('casual',axis=1).drop('registered',axis=1).drop('cnt',axis=1)\n",
    "\n",
    "# 丢掉一些不必要的特征\n",
    "X = X.drop('instant',axis=1)\n",
    "X = X.drop('dteday',axis=1) # 日期与输出y无关\n",
    "X = X.drop('weekday',axis=1) # 节假日、工作日对输出y影响重要，weekday意义不大\n",
    "\n",
    "# 两温度，相关性太强，取其一\n",
    "X = X.drop('temp',axis=1)\n",
    "\n",
    "# 用于后续显示权重系数对应的特征\n",
    "columns = X.columns\n",
    "\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当数据量比较大时，可用train_test_split从训练集中分出一部分做校验集；样本数目较少时，建议用交叉验证。在线性回归中，留一交叉验证有简便计算方式，无需显式交叉验证\n",
    "\n",
    "下面将训练数据分割成训练集和测试集，只是让大家对模型的训练误差、校验集上的测试误差估计、和测试集上的测试误差做个比较，实际任务中无需这么处理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将数据分割训练数据与测试数据\n",
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "# 根据作业要求，2011年数据作为训练样本，2012年数据作为测试样本\n",
    "X_train = X[X.yr==0]\n",
    "X_test = X[X.yr==1]\n",
    "\n",
    "y_train = data[data.yr==0]['cnt'].values\n",
    "y_test = data[data.yr==1]['cnt'].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.5、数据预处理 / 特征工程\n",
    "特征工程是实际任务中特别重要的环节\n",
    "\n",
    "scikit learn中提供的数据预处理功能：sklearn.feature_extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramFiles\\Anaconda2\\lib\\site-packages\\sklearn\\utils\\validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by the scale function.\n",
      "  warnings.warn(msg, DataConversionWarning)\n"
     ]
    }
   ],
   "source": [
    "# 将atemp hum windspeed 数据归一化处理\n",
    "from sklearn import preprocessing\n",
    "\n",
    "X_train = preprocessing.scale(X_train)\n",
    "X_test = preprocessing.scale(X_test)\n",
    "\n",
    "y_train = preprocessing.scale(y_train)\n",
    "y_test = preprocessing.scale(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3、确定模型类型\n",
    "3.1、尝试缺省参数的线性回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.635529</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.297763</td>\n",
       "      <td>season</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.007874</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.001075</td>\n",
       "      <td>mnth</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.033981</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.071831</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-0.111401</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.214388</td>\n",
       "      <td>weathersit</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       coef     columns\n",
       "6  0.635529       atemp\n",
       "0  0.297763      season\n",
       "4  0.007874  workingday\n",
       "1  0.000000          yr\n",
       "2 -0.001075        mnth\n",
       "3 -0.033981     holiday\n",
       "7 -0.071831         hum\n",
       "8 -0.111401   windspeed\n",
       "5 -0.214388  weathersit"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性回归\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 使用默认配置初始化\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 训练模型参数\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "# 预测\n",
    "y_test_pred_lr = lr.predict(X_test)\n",
    "y_train_pred_lr = lr.predict(X_train)\n",
    "\n",
    "# 看看各特征的权重系数， 系数的绝对值大小可视为该特征的重要性\n",
    "fs=pd.DataFrame({\"columns\":list(columns),\"coef\":list(lr.coef_.T)})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.1.1、模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of LinearRegression on test is 0.6720761991436093\n",
      "The r2 score of LinearRegression on train is 0.754471548519061\n"
     ]
    }
   ],
   "source": [
    "# 使用r2_score评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "# 测试集\n",
    "print 'The r2 score of LinearRegression on test is',r2_score(y_test,y_test_pred_lr)\n",
    "# 训练集\n",
    "print 'The r2 score of LinearRegression on train is', r2_score(y_train, y_train_pred_lr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0xb895b38>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAFsCAYAAADos0H8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHgBJREFUeJzt3X+cVXW97/HXJxylBEFgMJQIraxUFLgjQvYwiswSE3ucfh5Tupmo2eP0+5aaOXnsnjpanbyaZT/8kVamZfbLezTSY3VKAy+ihgaZFMhBxFSwTMHP/WOv4YzjDLOZ2TPznZnX8/HYj9l77e9a6/Ndi+G914/9nchMJElSeZ4z0AVIkqTOGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGkNeRHx5Yg4s3o+NyLW9PH6pkZERsRODVzm5ojYp1HL24H1nhIR66v1j++ndU6p1jeii/dbI+KKBq0rI+LFjViW1Bca9p+INFAi4n5gD2Ar8BTwn8DJmflngMw8eeCqa4zMHNXf64yIJuDzwOzMvKOT96cCfwQeryY9BHw5Mz/Tm/Vm5p+Afu+vVCKPpDVUvLEKsknAeuD/DHA9DdHIo/Ee2AMYCdzdTbux1bZ/M3BmRBze55VJw4QhrSElM58ArgH2a5sWEZdGxDmdtY+If4qI30XE5Or1URGxLCIeiYj/jIgDu1pXRMyKiCUR8Vh1SvjzHZocGxF/ioiHIuKMDvP9ulrHuoi4ICJ2bvd+RsSpEbESWNlu2ovb9efCiPhJRGyKiFsj4kXt5n9dRNwbEY9GxJci4j8i4j1d9GGXiPi3iHigevxbNW1f4N6q2SMR8fOutkObzFxCLdCnt1v+nhHxvYjYEBF/jIh/6m77dbxcEBF7V33YFBE3AhPaLeNZly8i4v6IeG0927rDfEdW/xY2RcTaiPhId32W+pohrSElIp4HvA34TR1tzwTeBbwqM9dExEzgG8BJwHjgK8API2KXLhbxReCLmbkb8CLgux3efyXwUmAe8MmIeHk1fSvwQWphM6d6/70d5j0GOIR2HzY6eAfwKWB3YBXw6apPE6h9SDmt6sO9wCu63AhwBjCbWrAeBMwCPpGZvwf2r9qMzczXbGcZVOueDRxQ1UNEPAf4EXAHsFfVzw9ExBHVLN1tvzbfApZS217/DCzsrpZ26tnWbb4OnJSZo6t+dPvBROprhrSGih9ExCPAY8DhwLnbaRvVUdsRwKszc0M1/UTgK5l5a2ZuzczLgL9TC7HOPAW8OCImZObmzOz4weBTmfm36nruHdRCkMxcmpm/ycwtmXk/tQ8Dr+ow779k5sOZ+bcu1v39zLwtM7cAV/LfR69HAndn5ver984H/ms72+JY4OzMfLDaDp8CjttO+848FBF/A34NfAn4QTX9YKA5M8/OzCcz8z7gq8Dbq/e7235ExJRqOWdm5t8z8xZqwV+XOrd1m6eA/SJit8z8S2beXu96pL5iSGuoOCYzxwK7AO8D/iMint9F27HAImpB+Gi76S8EPlydGn2kCv0XAHtGxLHVHcebI+L6qv0JwL7APRHx24g4qsN62ofjX6luhoqIfSPixxHxXxHxGPC/aXcKt/Lnbvrb6bKBPdvPm7W/oLO9u9n3BFa3e726mrYjJlTr/wgwF2iqpr+Q2rZrvz1Pp3atG7rffm31/SUzH283bXUn7TpV57Zu8w/UPuSsrk6vz6l3PVJfMaQ1pFRHwN+ndprzlV00+wtwFHBJRBzabvqfgU9n5th2j+dl5rcz88rMHFU93lCta2VmvgOYCHwWuCYidq2jzIuAe4CXVKd6TweiY1fq7XMH64DJbS8iItq/7sQD1MK0zZRq2g6ptvvngCf479PJfwb+2GF7js7MI6t56tl+64DdO0yf0u7548Dz2l5E7Wtbze3er2dbt/Xht5m5oKrnB3R9+l3qN4a0hpSoWUDtWu2Krtpl5s3UTvVeGxGHVJO/CpwcEYdUy9k1IuZHxOgu1vXOiGjOzKeBR6rJW+soczS10/KbI+JlwCl1da4+PwGmRcQx1Y1XpwJdnVEA+DbwiYhorq5nfxLozXeQPwP8r4gYCdwGPBYRH4uI50bEiIg4ICIOhvq2X2auBpYAn4qInSPilcAb2zX5PTCy2k9NwCeonU1pU9e2rpZ9bESMycynqnnq2ZdSnzKkNVT8KCI2U/vP9dPAwszc7leHMvNG4H9Suznsf1R3J58IXEDtaHsVtRvLuvJ64O5qvV8E3l7dXd6djwD/CGyi9sHgqjrmqUtmPgS8BfhXYCO1G8+WULu23plzqveXA3cCt1fTeuon1LbdiZm5lVqgTqf2feqHgK8BY6q29W6/f6R2E93DwFnA5W1vVJcr3lstdy21I+v2p/d3ZFsfB9xfnRY/GXhn3b2W+kjULllJGoqqO6zXAMdm5k0DXY+kHeORtDTERMQRETG2+upY2zXYbr+SJqk8hrQ09MwB/kDt9PIbqd353tVXuSQVzNPdkiQVyiNpSZIK1a+D90+YMCGnTp3an6uUJKk4S5cufSgzm7tr168hPXXqVJYsWdKfq5QkqTgRUdfIeZ7uliSpUIa0JEmFMqQlSSpUv16TliR17qmnnmLNmjU88UQ9I8tqsBg5ciSTJ0+mqamp+8adMKQlqQBr1qxh9OjRTJ06ldofL9Ngl5ls3LiRNWvWsPfee/doGZ7ulqQCPPHEE4wfP96AHkIigvHjx/fq7IghLUmFMKCHnt7uU0NakqRCeU1akkrU2trvyxsxYgTTpk1jy5Yt7L333nzzm99k7NixO7yq97znPXzoQx9iv/32e8b0Sy+9lCVLlnDBBRfs8DIBRo0axebNm+tqO3fuXM477zxaWlq2TVuyZAmXX345559/fo/WPxA8kpYkAfDc5z6XZcuWcddddzFu3DguvPDCHi3na1/72rMCugQtLS19HtBbt25t6PK6DemIGBkRt0XEHRFxd0R8qpq+d0TcGhErI+KqiNi5oZVJkgbMnDlzWLt27bbX5557LgcffDAHHnggZ511FgCPP/448+fP56CDDuKAAw7gqquuAmpHsW1DQF9yySXsu+++vOpVr+JXv/rVtuW9613v4pprrtn2etSoUQBs3ryZefPmMXPmTKZNm8Z11133rNrWrVvHYYcdxvTp0znggAP4xS9+UVefbr75Zo466igAWltbefe7383cuXPZZ599nhHeV1xxBbNmzWL69OmcdNJJ24L3lFNOoaWlhf3333/bNoDakNdnn302r3zlK7n66qvrqqVe9Zzu/jvwmszcHBFNwC8j4nrgQ8AXMvM7EfFl4ATgooZWJ0nqd1u3bmXx4sWccMIJANxwww2sXLmS2267jczk6KOP5pZbbmHDhg3sueee/OQnPwHg0UcffcZy1q1bx1lnncXSpUsZM2YMr371q5kxY8Z21z1y5EiuvfZadtttNx566CFmz57N0Ucf/YwbsL71rW9xxBFHcMYZZ7B161b++te/9qif99xzDzfddBObNm3ipS99KaeccgqrVq3iqquu4le/+hVNTU28973v5corr+T444/n05/+NOPGjWPr1q3MmzeP5cuXc+CBB26r+5e//GWP6tiebo+ks6btIkBT9UjgNUDbx6DLgGMaXp0kqd/87W9/Y/r06YwfP56HH36Yww8/HKiF9A033MCMGTOYOXMm99xzDytXrmTatGn87Gc/42Mf+xi/+MUvGDNmzDOWd+uttzJ37lyam5vZeeededvb3tZtDZnJ6aefzoEHHshrX/ta1q5dy/r165/R5uCDD+aSSy6htbWVO++8k9GjR/eov/Pnz2eXXXZhwoQJTJw4kfXr17N48WKWLl3KwQcfzPTp01m8eDH33XcfAN/97neZOXMmM2bM4O677+Z3v/vdtmXV07eeqOuadESMiIhlwIPAjcAfgEcyc0vVZA2wV59UKEnqF23XpFevXs2TTz657Zp0ZnLaaaexbNkyli1bxqpVqzjhhBPYd999Wbp0KdOmTeO0007j7LPPftYyu/oK0k477cTTTz+9bflPPvkkAFdeeSUbNmxg6dKlLFu2jD322ONZ3zM+7LDDuOWWW9hrr7047rjjuPzyy3vU31122WXb8xEjRrBlyxYyk4ULF27r67333ktrayt//OMfOe+881i8eDHLly9n/vz5z6hr11137VEN3anr7u7M3ApMj4ixwLXAyztr1tm8EbEIWAQwZcqUHpYpaUgYgDuWd3i9ja5xEBozZgznn38+CxYs4JRTTuGII47gzDPP5Nhjj2XUqFGsXbuWpqYmtmzZwrhx43jnO9/JqFGjuPTSS5+xnEMOOYT3v//9bNy4kd12242rr76agw46CKhdx126dClvfetbue6663jqqaeA2inziRMn0tTUxE033cTq1c/+i46rV69mr7324sQTT+Txxx/n9ttv5/jjj29I3+fNm8eCBQv44Ac/yMSJE3n44YfZtGkTjz32GLvuuitjxoxh/fr1XH/99cydO7ch69yeHfoKVmY+EhE3A7OBsRGxU3U0PRl4oIt5LgYuBmhpaek0yCVJHQzwh4UZM2Zw0EEH8Z3vfIfjjjuOFStWMGfOHKB2k9cVV1zBqlWr+OhHP8pznvMcmpqauOiiZ96WNGnSJFpbW5kzZw6TJk1i5syZ227COvHEE1mwYAGzZs1i3rx5245Ejz32WN74xjfS0tLC9OnTednLXvas2m6++WbOPfdcmpqaGDVqVJdH0vPnz982ZvacOXM49dRTu+33fvvtxznnnMPrXvc6nn76aZqamrjwwguZPXs2M2bMYP/992efffbh0EMPrX9j9kJkbj83I6IZeKoK6OcCNwCfBRYC32t349jyzPzS9pbV0tKSbXf8SRqGPJLu0ooVK3j5yzs7SanBrrN9GxFLM7Oli1m2qedIehJwWUSMoHYN+7uZ+eOI+B3wnYg4B/h/wNd3vHRJktSVbkM6M5cDz7pnPjPvA2b1RVGSJMkRxySpGN1dftTg09t9akhLUgFGjhzJxo0bDeohpO3vSY8cObLHy/APbEhSASZPnsyaNWvYsGHDQJeiBho5ciSTJ0/u8fyGtCQVoKmpib333nugy1BhPN0tSVKhPJKW1Lm++A6ypB3ikbQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklSonQa6AEn9rLV1oCtonKHUF6kTHklLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSpUtyEdES+IiJsiYkVE3B0R76+mt0bE2ohYVj2O7PtyJUkaPur5nvQW4MOZeXtEjAaWRsSN1XtfyMzz+q48SZKGr25DOjPXAeuq55siYgWwV18XJknScLdD16QjYiowA7i1mvS+iFgeEd+IiN27mGdRRCyJiCUbNmzoVbGSJA0ndYd0RIwCvgd8IDMfAy4CXgRMp3ak/bnO5svMizOzJTNbmpubG1CyJEnDQ10hHRFN1AL6ysz8PkBmrs/MrZn5NPBVYFbflSlJ0vBTz93dAXwdWJGZn283fVK7Zm8C7mp8eZIkDV/13N19KHAccGdELKumnQ68IyKmAwncD5zUJxVKkjRM1XN39y+B6OStnza+HEmS1MYRxyRJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqVLchHREviIibImJFRNwdEe+vpo+LiBsjYmX1c/e+L1eSpOGjniPpLcCHM/PlwGzg1IjYD/g4sDgzXwIsrl5LkqQG6TakM3NdZt5ePd8ErAD2AhYAl1XNLgOO6asiJUkajnbomnRETAVmALcCe2TmOqgFOTCxi3kWRcSSiFiyYcOG3lUrSdIwUndIR8Qo4HvABzLzsXrny8yLM7MlM1uam5t7UqMkScNSXSEdEU3UAvrKzPx+NXl9REyq3p8EPNg3JUqSNDzVc3d3AF8HVmTm59u99UNgYfV8IXBd48uTJGn42qmONocCxwF3RsSyatrpwGeA70bECcCfgLf0TYmSJA1P3YZ0Zv4SiC7entfYciRJUhtHHJMkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklSoev6etKTStbYOdAWS+oBH0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKtROA12ANKS0tja23WAwlPoCw3MfqlgeSUuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYXqNqQj4hsR8WBE3NVuWmtErI2IZdXjyL4tU5Kk4aeeI+lLgdd3Mv0LmTm9evy0sWVJkqRuQzozbwEe7odaJElSO725Jv2+iFhenQ7fvWEVSZIkoOcjjl0E/DOQ1c/PAe/urGFELAIWAUyZMqWHq5OkwjgymfpBj46kM3N9Zm7NzKeBrwKzttP24sxsycyW5ubmntYpSdKw06OQjohJ7V6+Cbirq7aSJKlnuj3dHRHfBuYCEyJiDXAWMDciplM73X0/cFIf1ihJ0rDUbUhn5js6mfz1PqhFkiS144hjkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVaqeBLkDSdrS2DnQFkgaQR9KSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVyhHHNHzVO5qXo35pqPN3oVgeSUuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKlS3IR0R34iIByPirnbTxkXEjRGxsvq5e9+WKUnS8FPPkfSlwOs7TPs4sDgzXwIsrl5LkqQG6jakM/MW4OEOkxcAl1XPLwOOaXBdkiQNez0dcWyPzFwHkJnrImJiVw0jYhGwCGDKlCk9XJ0kDXE7MpqXI38NG31+41hmXpyZLZnZ0tzc3NerkyRpyOhpSK+PiEkA1c8HG1eSJEmCnof0D4GF1fOFwHWNKUeSJLWp5ytY3wZ+Dbw0ItZExAnAZ4DDI2IlcHj1WpIkNVC3N45l5ju6eGteg2uRJEntOOKYJEmFMqQlSSqUIS1JUqEMaUmSCtXTEcckSfXoi9HB6l2mI5MNeh5JS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChHHJMGgiNBSaqDR9KSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQhnSkiQVypCWJKlQhrQkSYUypCVJKpQhLUlSoQxpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmF2mmgC5CK19o60BVIGqY8kpYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVKhefQUrIu4HNgFbgS2Z2dKIoiRJUmO+J/3qzHyoAcuRJEnteLpbkqRC9fZIOoEbIiKBr2TmxR0bRMQiYBHAlClTerk6DTmO5iVJXertkfShmTkTeANwakQc1rFBZl6cmS2Z2dLc3NzL1UmSNHz0KqQz84Hq54PAtcCsRhQlSZJ6EdIRsWtEjG57DrwOuKtRhUmSNNz15pr0HsC1EdG2nG9l5v9tSFWSJKnnIZ2Z9wEHNbAWSZLUjl/BkiSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhTKkJUkqlCEtSVKhDGlJkgplSEuSVChDWpKkQvX470kXobW1b9pKknqn3v9z/b95uzySliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCDe8SxwWKgRt4ZyBHZHEVIGnj+Xm/fIBgVzSNpSZIKZUhLklQoQ1qSpEIZ0pIkFcqQliSpUIa0JEmFMqQlSSqUIS1JUqEMaUmSCuWIY6oZaiMJSRocGj3q1xD7v8wjaUmSCmVIS5JUKENakqRCGdKSJBXKkJYkqVCGtCRJhepVSEfE6yPi3ohYFREfb1RRkiSpFyEdESOAC4E3APsB74iI/RpVmCRJw11vjqRnAasy877MfBL4DrCgMWVJkqTIzJ7NGPFm4PWZ+Z7q9XHAIZn5vg7tFgGLqpcvBTYCD/W44sFtAsO372D/7b/9t//DU2d9f2FmNnc3Y2+GBY1Opj0r8TPzYuDibTNFLMnMll6sd9Aazn0H+2//7b/9H579703fe3O6ew3wgnavJwMP9GJ5kiSpnd6E9G+Bl0TE3hGxM/B24IeNKUuSJPX4dHdmbomI9wH/DowAvpGZd9cx68XdNxmyhnPfwf7b/+HN/g9fPe57j28ckyRJfcsRxyRJKpQhLUlSofo8pCPi3Ii4JyKWR8S1ETG2i3ZDbojRiHhLRNwdEU9HRJe330fE/RFxZ0Qsi4gl/VljX9qB/g+5fQ8QEeMi4saIWFn93L2Ldlurfb8sIgb9zZfd7c+I2CUirqrevzUipvZ/lX2jjr6/KyI2tNvf7xmIOvtKRHwjIh6MiLu6eD8i4vxq+yyPiJn9XWNfqqP/cyPi0Xb7/5PdLjQz+/QBvA7YqXr+WeCznbQZAfwB2AfYGbgD2K+va+uHvr+c2gAuNwMt22l3PzBhoOsdiP4P1X1f9e1fgY9Xzz/e2b/96r3NA11rA/vc7f4E3gt8uXr+duCqga67H/v+LuCCga61D7fBYcBM4K4u3j8SuJ7aOBuzgVsHuuZ+7v9c4Mc7ssw+P5LOzBsyc0v18jfUvk/d0ZAcYjQzV2TmvQNdx0Cps/9Dct9XFgCXVc8vA44ZwFr6Sz37s/12uQaYFxGdDY402Azlf8t1ycxbgIe302QBcHnW/AYYGxGT+qe6vldH/3dYf1+Tfje1T1Ed7QX8ud3rNdW04SKBGyJiaTWM6nAylPf9Hpm5DqD6ObGLdiMjYklE/CYiBnuQ17M/t7WpPsA/Cozvl+r6Vr3/lv+hOtV7TUS8oJP3h7Kh/PterzkRcUdEXB8R+3fXuDfDgm4TET8Dnt/JW2dk5nVVmzOALcCVnS2ik2mD4rth9fS9Dodm5gMRMRG4MSLuqT6RFa8B/R+0+x623/8dWMyUav/vA/w8Iu7MzD80psJ+V8/+HNT7fDvq6dePgG9n5t8j4mRqZxRe0+eVlWOo7vt63U5tzO7NEXEk8APgJduboSEhnZmv3d77EbEQOAqYl9WJ+Q4G7RCj3fW9zmU8UP18MCKupXbabFCEdAP6P2j3PWy//xGxPiImZea66pTeg10so23/3xcRNwMzqF3bHIzq2Z9tbdZExE7AGBp8inCAdNv3zNzY7uVXqd2nM5wM6t/33srMx9o9/2lEfCkiJmRml394pD/u7n498DHg6Mz8axfNhu0QoxGxa0SMbntO7Ua7Tu8MHKKG8r7/IbCwer4QeNaZhYjYPSJ2qZ5PAA4FftdvFTZePfuz/XZ5M/DzLj68Dzbd9r3D9dejgRX9WF8JfggcX93lPRt4tO2S0HAQEc9vu/8iImZRy+CN252pH+52W0XtGsSy6tF2V+eewE/btTsS+D21I4gzBvouvQb1/U3UPjn+HVgP/HvHvlO7E/SO6nH3UOl7vf0fqvu+6td4YDGwsvo5rpreAnytev4K4M5q/98JnDDQdTeg38/an8DZ1D6oA4wErq7+b7gN2Gega+7Hvv9L9Xt+B3AT8LKBrrnB/f82sA54qvrdPwE4GTi5ej+AC6vtcyfb+dbLYHzU0f/3tdv/vwFe0d0yHRZUkqRCOeKYJEmFMqQlSSqUIS1JUqEMaUmSCmVIS5JUKENakqRCGdKSJBXq/wOSqwyS0c1yBQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xba49550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f,ax=plt.subplots(figsize=(7,5))\n",
    "f.tight_layout()\n",
    "\n",
    "ax.hist(y_train - y_train_pred_lr,bins=40,label='Residuals Linear', color='r', alpha=.5);\n",
    "ax.set_title(\"Bike-sharing of Residuals\")\n",
    "ax.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "残差分布和高斯分布比较匹配，左边有噪声，待分析。残差原理还不理解，待掌握。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl4lOX18PHvyWSysCayEzYt/Ghlh2ix/iyIUlBUlqKo6CvVYm2t2qJUXFBptS5Yl7ZvrRStilpRwbwCVRG3ihUEBBIRaBUEicoeEAjJJDnvHzMDk2Rm8iSZmSeTnM915WImmTzPSUhO7vXcoqoYY0w8pLgdgDGm8bIEY4yJG0swxpi4sQRjjIkbSzDGmLixBGOMiRtLMMaYuLEEY4yJG0swxpi4SXU7gNpo27at9ujRw+0wjGny1qxZs0dV29X0uqRKMD169GD16tVuh2FMkyci25y8zrpIxpi4sQRjjIkb1xKMiGSIyEcisl5ENojILLdiMcbEh5tjMCXACFU9JCJeYLmIvKaqK1yMyRgTQ661YNTvUOCpN/BmxWmMcUFpaWlcruvqGIyIeERkHbALeFNVV4Z5zdUislpEVu/evTvxQRrTyOXn5zNkyBCef/75mF/b1QSjquWqOhDoApwqIn3DvGaOquaqam67djVOuxtjHKqoqOChhx7ilFNOYc+ePbRp0ybm92gQs0iqWgS8C4x2ORRjmoQdO3YwcuRIbrzxRs455xzy8/MZNWpUzO/j5ixSOxHJCjzOBM4GNrkVjzFNyapVq1i5ciVz587llVdeIV69AzdnkToBT4uIB3+ie1FVF7sYjzGN2oEDB/j3v//NOeecw/jx49myZQvt27eP6z1dSzCqmg8Mcuv+xjQl77//Ppdffjm7d+9m27ZttG3bNu7JBRrIGIwxJj5KS0u57bbbGD58OB6Ph2XLltG2bduE3T+pNjsaY5zz+XycccYZfPTRR1x11VU8/PDDtGzZMqExWAvGmEbK6/Uyfvx4FixYwNy5cxOeXMASjDGNys6dOxk7dixvv/02ADNmzGDChAmuxWMJxphGYtGiRfTr14833niD7du3ux0OYAnGmKR3+PBhrrnmGi644AI6d+7MmjVrmDJlitthAZZgjEl68+fPZ86cOfzmN79h5cqV9OnTx+2QjrFZJGOSUHl5ORs3bqRv375MmTKFAQMGMGTIELfDqsZaMMYkma1btzJs2DDOOOMM9uzZQ0pKSoNMLmAJxpikoao888wzDBgwgIKCAv785z/HZQd0LFkXyZgk4PP5mDx5Mi+99BI//OEPeeaZZ+jevbvbYdXIWjDGJAGv10vLli259957efvtt5MiuYC1YIxpsI4ePcrMmTO54oor6Nu3L3PnzkVE3A6rVizBGNMAFRQUMHnyZAoKCmjfvj19+/ZNuuQC1kUypkGpqKjg4YcfJjc3l507d7JkyRKmT5/udlh15mZFu64i8o6IbAyci3SDW7EY01DMnTuXadOmMXr0aAoKCjj33HPdDqle3OwilQE3qurHItISWCMib6rqpy7GZIwr9u7dS5s2bbjiiito3bo1F110UVJ2iapy81ykr1X148Djb4GNQI5b8RjjhoMHDzJlyhQGDx7MgQMHSE9PZ9KkSY0iuUADGYMRkR74y2fauUimyVi+fDkDBgxg3rx5TJkyhWbNmrkdUsy5nmBEpAWwAPiVqh6s+nE7F8k0NmVlZdx+++0MGzaMlJQUli9fzqxZs/B6vW6HFnOuTlMHzqReADynqgvdjMXEV97aQma/sZmviorpnJXJ9FG9GTeoafaIU1JS+PDDD7niiit49NFHXak0lyiuJRjxdzKfADaq6kNuxWHiL29tIbcsLKDYVw5AYVExtywsAGgySUZVeeKJJzjnnHPIyclhyZIlZGRkuB1W3LnZRToduBwYISLrAm/JPSdnqslbW8iNL64/llyCin3lzH5js0tRJdauXbsYO3YsU6dO5bHHHgNoEskF3D0XaTnQOIbKTVjBlku5atiPf1VUnOCIEm/JkiVceeWVHDhwgEceeYTrrrvO7ZASyrYKmLiZ/cbmai2XUJ2zMhMYTeI9/fTTTJkyhf79+/PWW2/Rt29ft0NKOEswJm6itVAyvR6mj+rdKAd/y8rKSE1NZezYscyaNYubb76Z9PR0t8NyhSUYEzedszIpDJNkPCLcO6EfQMTBXyDpEk95eTkPPPAAeXl5vP/++2RlZXHHHXe4HZarXF8HYxqv6aN6k+n1VHpfptfDHy4awLhBOWG7UMW+cu56dQO3LCygsKgY5XjiyVtbmMDoa+eLL75g+PDh3HrrrfTo0YOjR4+6HVKDYAnGxM24QTncO6EfOVmZCJCTlcm9E/oda4lE6kIVFfuSZtZJVZk3bx79+/cnPz+fefPm8cILL9CqVSu3Q2sQrItk4mrcoJyIXZtIXahIoo3phBvLgfh3s3w+Hw888MCxJf89evSI6fWTnaMEIyLdgV6qukxEMoHUwAZFY+ps+qjelcZgwN+FyvCmsP+Ir9rrI8063Z5XwHMrthOcDC8sKuZX89dVek2sF/e99957DBo0iFatWrF06VLat2+Px+Op+RObmBq7SCIyFXgZeDzwri5AXjyDMk1DpC7UmP6dqi2QCs46VXV7XgHPhiSXaILjO/VRUlLCTTfdxPDhw/n9738PQKdOnSy5ROCkBXMtcCqBnc6q+l8RaR/XqEyTUbULlbe2kAVrCislDAF+PKR6VytvbSHPrajdGcxFxT7y1hbWqRXzySefMHnyZPLz8/n5z3/OzJkza32NpsZJgilR1dJgfQoRSQVHfzCMqbVwM0sKvLOpeqmO2W9srtMP4q/mr2P1tn3kdj/B8RjNwoULufTSS2ndujWLFy9mzJgxdbhz0+MkwbwnIrcCmSIyEvgFsCi+YZmmKtKgb7gB3toMEFf17IrtPBvS+iksKubXURJPbm4u3x9xDkcGX8Yv34ffF7ydFGtz3OYkwcwArgIKgJ8B/wTmxjMo0zTlrS1ECN88DjfA6xGJuM+pLhR/4pn/0Zf4KpTDmz9g/X8+ZMah3zAxtwu7hvysSe8Ir4saE4yqVgB/C7wZEzfRujyHS8o4ccYSWmd6EYGiI7649dNLig+zb9kcDn+yjLSOvTj87QGeX1lBRZUbBtfmWIKJrMYEIyJbCfNHRVVPiktEpsmK1uUpKvZV+jdeju7YyN4lf6DswC5anzaJ1qdfgnhSqyWXoKawI7w+nHSRckMeZwAXAifEJxzTlMW6y1NbWu5jz6LZIEKHS+8jo8vJNX5OiggnzliSNPulEs1JF2lvlXc9IiLLgXrv4hKRJ4HzgF2q2vT2sptK3EouvqJvSG3ZFvF4af/jmaS27kBKurMC3MGYbUwmPCcL7QaHvOWKyDVArIqIPgWMjtG1TJLLqWN9mLp+nqry7brX+frJazmw4iUA0tqf6Di5VBVpv1Te2kJOv+9tTpyxhNPve7tBb9qMNSddpD+EPC4DvgAuisXNVfVfgSNLjGH6qN5Mf2k9vkgDHmEEV/jOfmNzraaty48cYO9rf6T4s5VkdB9Ii/4j6xJyNVVjyFtbyPSX1+MrP97Smf7yeqBptHRqbMGo6pkhbyNVdaqqNrxtrSbpjRuUQ4uM6H/zsjK9ZDfzVtudHa40RCRHt+Xz1ZPXUrz1Y7JHTKX9pN+S2rJtDL4C/6rj0BbKrEUbjiWXIF+5MmtR/bYsJIuI/5siMi3aJybqJAARuRq4GqBbt26JuKVxUVGYTY5B2c28rL3jRxE/nuFNiVqiMyglowWpLdvRZtLdpLXrUZcwI1KoNHUdbtNmtPc3NtH+XDSIw1pUdQ4wByA3N9e2KDQi4UosRCvhsP9I+H1EVbsh4ZR88xnFn68i6/RLSOtwEh3/z0NxO57Vpq6Pi5hgVHVWIgMxTUuks5J+PCSnUumFqsItbAvXDQnSinIOfrSQovefxdMsi5aDx+DJbBXXs59DVx1nZXrDrt3Jymx8pziG42QWKUNErhWRv4jIk8G3WNxcRP4BfAj0FpEdInJVLK5rGr5I5TLf2bSbyUMjd4XDtQ4idTfKDuxi5wu3UfTe0zTrNZROV/4ZT2Z8K80JVCorcdcFffCmVE5m3hThrgv6xDWOhsLJLNI8YBMwCvgtMBnYGIubq+olsbiOST6RuhFfFRVz97h+LFizg2JfRbWPOz3qRMt9fPPczVSUHKLNmF/TvM+IuLZawJ9cJg/tVqmFFXycbAXMY8VJgumpqheKyFhVfVpEngfeiHdgpnGLNNbSOSuTvLWFlIWZqvamCGd+tx2n3/d2pV/W0G5IRckRJC0T8XhpM/qXpJ6QgzerY9y/npwoiSNa2dDGzkmCCbY/i0SkL/AN0CNuEZkmIVK5zOCalnBjKmmpKSxYU1hp3ObX89fRs31ziop9HN2ez57FD9P6tAtpOehcMk8aEvevQwQevmig4wTSGM+BisbJqQJzRCQbmAm8CnwK3B/XqEyjF+3EgUjdp8Ol5WGLUf3nqyL2v/MkO/9xG+JNI61jz/h/AcH7K46PVAkObCfTcSz15aQF83dVLQfeA2wHtYmJaH/Ja3PaQOme7exZ9CC+XVtoMXA02Wf+lJS0xB4sH65sQ7ivL9LAdmMu+eCkBbNVROaIyFkS71Ey0yTU9Jd8+qje1Yp+R1J+cDflh/bR7sczaTPqlwlPLkGhra5IX19tqvU1Fk5aML2B8/EX/35SRBYBL6jq8rhGZhq0qn+hz/xuO97ZtNvR2EJNf8nHDcqpduxIqLJD+yjZnk/zk4eTedIQcn4217XEEhQ6uxXp64tUjsLpzFgyclKuoRh4EXgxMBbzKP7ukp3T0EjUduAx3CK5qvVto23oizZFHZQToZt05D//Zu/rf0bLfWScOBhPZivXk0vVI1UifX3lqmR6PWEHthsrpwevDQMmAecAq4jRbmrjvkgraqF6cggmIifjI8ENfeESTKQxFgV6zFhCdjMvR6u0ACpKjrDvrb9xuOBN0jr2pO15N8V90RzUXAQr3PR0pK8vJ2QspqnMIjktmbkOfytmuqoejntUJmGcDjxWTURORFphW1NZhqqfp+U+vn5mGmX7Cml12kVknX4J4knMUvsKVR6ZNDDslHroOduhok3BN7U1MU5aMANU9WDcIzGucNJdgfCJqLZq0wICUK1AJAXxeGl1yji8bbqQ0TWxhQ87Z2XWejVuU1+9G8rJGIwll0Ys2oraUHWZ6Qjd0FfbFpBv/1fsWfwHWp82iWY9T6XlwLoVPsxu5q1zaYTQ8ZHatjyaWkslEkdjMCa+3FzdGa05Hyra2pSsTC8Hj/oqVd73pgjnDejE6fe9XatKc6rKofw32f/WHCTFAxX1azUdLC7zT3mLf1GcU9GW/hvnLMG4rDaDrPHgtDkfKREFxyHy1hYya9GGY62FVI8wf9WXUWu0VFV+5AB7X/8Txf9dQUb3/rQ5dxqprepXae7YAG0tKwl9MGNEve5r/Bp8RbvGriGs7nTSnHeSiI6G7H4OtxO6JsVb1lC8ZTXZZ15Jy1PGIeJkHWjsZTdrGrVaEsFJRbvewCn49yGBf9Hdv+IZVFPidJAV3OlKOb1nXQeBK3wllO78nIwuJ9O8z5mkdzk5Ibufo7nz/KZRqyURaqxoJyJLgcGq+m3g+V3ASwmJrglwOsjqRleqNvesyyBw6c7P2bPoQcq+3UPONU/gyWzlenLJyvTauEsMOWmDdgNKQ56XEqNyDSIyWkQ2i8hnIjIjFtdMNuGq4YcbZI3WlYqXSPe88cX11XYA12a5u1aUc2DFy3z9zI1UlBym3bhb67VozhPYIuepxVa55mnVF6Jnej0RK8015bON6sNpRbuPROQV/ENl44Fn6ntjEfEA/xcYCewAVonIq6r6aX2vnUycDrLWpisVK9GWvE9/eT13vbqBA8W+Y3uR5n/0ZY1nGmm5j50v3kHJ9gKa/c8POGH0L+ucXDwpQgocu2dwKX6GN6XGqenDpeVkN/OiyrGvIVL3z+2B+GTmZB3MPSLyGnBG4F0/UdW1Mbj3qcBnqroFQEReAMbirzfTpDgZZHXalYqlaFPTvnI9VkWusKiY51Zsx+upuQUhHi/pnf6HFn3Ponnfs+pcxrJ5mocjpeVUHfUp9pWTnpqCN0VqTHb7j/jI9Hp4eFL0glENYSA+WTkdpm8GHFTVR4EdInJiDO6dA3wZ8nxH4H2ViMjVIrJaRFbv3r07BrdNTk67UrG+p5OkAf6mbWmEKemKo4fYs+QhSr7yd+eyh/+EFv3OrleN3MOl5RFnnouKfY5Ph3TSzXSj9dhYODlV4E7gZuCWwLu8wLMxuHe4n65qPxWqOkdVc1U1t127djG4bXKKVgEukvqOG4wblEPztPotlTq6vYCv/n4dhze8S+muLXW6RryLENWUKCK1EhtzmYVYcfLTMx4YBHwMoKpfiUgsDmXbAXQNed4F+CoG1220arP8vLbjBiMfepf/7jq+j7VDyzRSPZ6wZ/o4oeU+it5/joMrF5Ca3ZGOl80mvXPdWlutw6wUjqWaEoXT1c6mOicJplRVVUQUQESax+jeq4Bege5WIXAxcGmMrt0kha5ZSQlTZiDSuEHV5AKw89tS6sKbAr4KOJT/JgdXvkyLAaPIHvFTUtLq/te+rknOCSeJwjYv1p2TBPOiiDwOZInIVOBKYG59b6yqZSLyS/xHoHiAJ1W1aZwIHgd5awsrlUCIVMMkXHeganKpK1WleP8uUlt3oMWAUaSekENm9wExuXYseUSoUK1VorDNi3XjZBbpQREZCRzEv6r3DlV9MxY3V9V/Av+MxbWaurte3eBoYDNe4wZlh/ax95+PUrrrczpf9RiezJYJTS7BwxNr+hZ4PcLsiQMsWSSIk4JT96vqzcCbYd5nGggn3Yh4jRsc+c+H7H39T6jvKNlnXkVKRouY3yMaEXjoooGAP9FG+l5kN/Ny5/l9LLkkkJMu0kj8s0ihzgnzPpNAVfcIOZGeGn7SsFf75nXqJmm5j31LH+NQ/lLSOnyHtufdhLdt15o/McZUj4+TWPJoWCJOU4vIz0WkAPiuiOSHvG0FChIXoqkq3LEYThQV+8Ie9PXmtOGOF0RVkpJK+dFvaTV0Ih0vfzCuycXpehzTsET7uXoe/87p/xf4N/g2RFUnJyA2E0F9yldGWljmtLiCVpRz4N/z8RV9g4jQbtwtZA+bUucaudnNvDWuc8nJymT2xAGVKuSFivR+475ou6kPAAdE5FFgX8hu6pYi8n1VXZmoIE1l0RaG5WRlHus2OT3oK29tIULNNZl8+79mz+IHKf1qM4jQ+rSL6lWz5bKh3bh7XD+AiJXvcrIyKxV/qlos3JsiETcoGvc5+el4DDgU8vxw4H3GJZHGXIK/jFvvG8MHM0aQE+F1WVUKKs1+Y3PU5BIsY/n1U9fj27uDtudPp/Vpzk6uOf07J1Tb4iBUTi5Q81aI4JiTr0KP7ZrOycpk9oU2I9SQOUkwonp8UYWqVmClNl3ldF/S9FG98aRU74DsP+Jj4Kylx8ZiahrDObT+dfa+9ihpHXvS+co/0fzkYY7izG7m5bmpp1Xb4vDwpIGVkgtE3woROuYEx3dN22K3hs9JotgiItdzvNXyC6Bum0pMTISuLC0sKsYjUmlsJfSXrjzCwpDggO/qbfsi3qfCd5QUbwbNTz4TVGkxcHStukRj+nc6Fk99FrPZbubk5STBXAP8Ebgdfzf9LeDqeAbVWMSzxGXwOuH2G63eto93Nu2usWVS7CuvdORrUIWvhKL3nuLoF+voeMXDpKRl0HLQubWO8Z1Nsdn9bruZk5eTlby78O8TMrWQiCJFkf6yP7die22L6B9TunMLexY9iG/vdloOuaBeg7ixSgBu1MIxsRHtVIHfqOoDIvInwpdRuD6ukSW5eDbr89YWcuvCfI5EqNxfl+SiWsHBj16h6F/z8DRrRfuLfkvmiYPrFWesEoDtZk5e0VowGwP/rk5EII2Nk2Z9XbpQeWsLmfbiutiXLqio4MjmD2jW89R6lbEMimUCsN3MySvaOphFgX+fTlw4jUdNzfq6dqFmv7E5psnl8KblZHTvjyezFR0m/Q5Ja+ao0ly02rcekRqLYdWW7WZOTtG6SIuI0tpW1QviElEjEa5ZL8CZ3/VX5YtWsR8iJ5lYjWtUlBxm39LHOPzpu7Q6bRLZP7yclHRnpX6Cx6oCUU97NCZaF+nBwL8TgI4cL5N5CfBFHGNqFMYNymH1tn2VBlwVWLCmkNzuJ0St2B+tJdM601vvAkxHv/yEPYv/QPm3e2n9v5MdL5oLqto9sa6LiUS0hhPBReRfqvrDmt5Xq5uKXAjcBXwPOFVVHY3z5Obm6urVyTMkFG35O0Rf4FZ1iTwEikq9vL5W5z1XdWjDO+xd/BCpWR1pe96NpOd8t9bXCBebaVpEZI2q5tb0OifrYNqJyEkhx4ucCNS3+vYn+FtGj9fzOg1atIHeyUO7RZ1OLiwqZtBvl1J05Pi5Q/9Y+WXESnU1UVVEhMweA2mZewFZZ1xW5zKWtv7EOOUkwfwaeFdEgqt3ewA/q89NVXUjUK9jKxqKcDNBEH1/T1YzLwvWFNY4nRwcQC0sKg67IM4JVeXQ2iUc+e9K2l94F57m2Zxw1tQ6XSvI1p8Yp5wstHtdRHoBwbb0JlUtiW9Yx4nI1QRWDnfr1i1Rt3Uk3EzQ9JfWgxCxG5Pp9aBKncst1Eb5of3see0Rjm5ZQ8aJQ1DfUcThQG5WppfzBnRiwZpCW39i6sxJycxmwDSgu6pOFZFeItJbVRfX8HnL8A8OV3Wbqv4/pwGq6hxgDvjHYJx+XiKEmwmqqS5uempKXKvkBx357wr2vvZH1HeUE0ZeQ4tBYxy1GAXYet+YY89zu59gg7imzpx0kf4OrAFOCzzfAbwERE0wqnp2/UJr+OoyFpGI5KJlPva//QSelm1pe/5NpLV13vKr2v2x9SemPpwkmO+o6iQRuQRAVYulMQyexEC0ok5uKPnmM7xtupLiTaf9pN+R2rJNrSrNWffHxJqTnWylIpJJYNGdiHwHqNcYjIiMF5Ed+FtFS0Tkjfpczy3h6rJ4U8Rx/dhIBaFqSyvKKfr3C3zzzDQOfviiP46sjrUuY2kL5EysOWnB3Am8DnQVkeeA04Ep9bmpqr4CvFKfazQEkfbIhL4v2ojMBzNGRFwr45Sv6Bv2Lv4DJYUbafa9YbQ6dXydrpOTlWnJxcRc1AQT6Aptwr9mZSj+McAbVHVPAmJLCuHGKEKr9qdI+MPAsgNlK6eP6s2v56+r0w7oI599xJ5Fs0FSaHv+TTQ/eXgdrmJdIxM/URNM4EzqPFUdAixJUExJrerUdbh1cV6PcOf5/kLV4bYUOOXN7kR6l5Np86NrSW3d3vHnXTa0G+9s2m0zQybunHSRVojIKaq6Ku7RNAKRjhSJdh7y3eP6HZsOrrEK3daPKf58FdlnXY23TVc6XDirVvF5RKrVwzUmXpwkmDOBa0TkC/wnCgj+xk3/eAaWrCIliHJVvghZX1JVMOHc+OL6sNsBtKyU/e8+xbdrXsXbpisVJYfx1OGI1rpuNTCmLpwkmHPiHkUj4hEJ+0scprh/JbfnFUTsJpXu2sqeRbPx7dlOyyHnkzVsCine9DrFF6uZK2OciFYPJgN/we+e+I+KfUJVyxIVWLKK1EKoUP/4TLClkre2MOpB7UFa5mPXS3eBVtD+wllknjSkXvHZYK5JpGgtmKcBH/A+/lbMycANiQgqmeVEWXwXrMebt7aQafPXRT2utezQPjzNs5BUL23HzsB7Qmc8zVpHfL3grxVTWlYesVbvZUO72WCuSahoC+1OVtXLVPVxYCJwRoJiSmrRWgjBrQV3vbohanI5vPF9vp77c75dlQdARpfvRU0uQc3TU/n9hP48MmngsWlw8G9cfCTMYWfGxFu0FsyxtruqltnuAGfGDcqJ2PVJEeHEGUsiTkdXlBxm35t/5fCGd0jr1JvMXkMd31c5Xtf33gn9WHvHj+r2BRgTQ9FaMANE5GDg7Vugf/CxiBxMVIDJ6K4L+lTbQgD+8ZlIyaWkcCNfPXkdhz99j9anX0LHyx7Am9251vcOPeHRGLdFO1Wg+m+IcaTqFoKUCDNLobSiHPGk0nHy/aTnfK/SxzK9nlrVj7GKc6ahqPuxfSaqcYNy+GDGCLbeN4aKCMnFt/dLDq5ZBEBG1750/uljx5KLR6TSIfC1mV62inOmoXCyDsbUU9WyDqrKoXWvsf/tJ5C0DJr3ORNPRgskxd9ojHT0R9UjQsKxfUWmIbEWTAKElnUoP7yf3Qt+y76lfyG9ax86/eRPlVbkBlssVZPLuEE5NbZk4nHgmTH1YS2YBAj+wt+/pIBVj02j/HAR2Wf/jJaDx1Q6XL6m40CCO7erbqgEO/DMNEyuJBgRmQ2cD5QCnwM/UdUiN2JJhJKSkmPJofen1/BtZkfS2nWv9jqng7N2VrNJFm51kd4E+gY2TP4HuMWlOOJu9erV9O/fn+effx6A+6f/jPQwyQVqNzgbOoj8wYwRllxMg+RKglHVpSH7mlYAXdyII57Ky8u55557OO200zhy5AidO/vXtIwblMPkod2oumzRBmdNY9QQBnmvBF6L9EERuVpEVovI6t27dycwrLrbunUrw4YN4/bbb2fixInk5+czfPjwYx+/e1w/Hp40kJyszEpT0dYKMY1N3MZgnJyLJCK3AWXAc5Gu05DPRYpkzZo1FBQU8Oyzz3LppZeGPY/IjgMxTUHcEkxN5yKJyBXAecBZqslfBWnfvn18+OGHjBkzhokTJzJ8+HDatm3rdljGuMqtWaTRwM3AMFU94kYMsbRs2TImXXoZRQcOkvPzv9O1Yzv/rI7lF9PEuTUG82egJfCmiKwTkb+6FEe9HD16lGnTpjFy5EgOVaTR/pJ7SclocWxXc+jpAsY0Ra60YFS1pxv3jaWSkhKGDh3K+vXr6Th0LN4fXE6KN+PYx4O7mm2cxTRlDWEWKakEh4vS09O55JJLWLJkCRnDplZKLkG2q9k0dZZgaqGwsJDRo0fz3nvvAXDzzTdz7rnnRlwgZ7uaTVNnCcahl19+mX79+rF8+XIkIEw6AAAHr0lEQVS+/vrrSh8Ld0a1LZwzxhJMjQ4ePMiUKVO48MIL6dmzJ+vWrePiiy+u9JrQnc62cM6Y42w3dQ1eeOEF5s2bx8yZM5k5cyZerzfs62zhnDHVWYIJw+fzsXHjRvr3789Pf/pTvv/97zNgwAC3wzIm6VgXqYrNmzfzgx/8gOHDh7N//35SUlIsuRhTR5ZgAlSVxx9/nMGDB7Nlyxb+9re/kZ2d7XZYxiQ16yLhXzQ3ceJEFi9ezMiRI3nqqaeOlVcwxtSdtWDwL5rr0KEDjzzyCK+//rolF2NipMkmmCNHjnD99dezYcMGAObOncsNN9xASkqT/ZYYE3NNsou0Zs0aJk+ezObNm+nVqxd9+vRxOyRjGqUm9ee6vLyce++9l6FDh3Lo0CHeeustrrvuOrfDMqbRalIJ5rHHHuPWW29lwoQJ5OfnM2JE5CNCjDH11+i7SKrKnj17aNeuHVOnTqVz586MHz8+bBlLY0xsudKCEZHfiUh+oNjUUhGJy7TNvn37uPjiiznllFM4ePAg6enpTJgwwZKLMQniVhdptqr2V9WBwGLgjljf4K233qJ///4sXLiQa665hubNm8f6FsaYGrh1LtLBkKfNgZgV/fb5fNx0002cffbZtGjRghUrVjBjxgw8Hk/Nn2yMiSnXBnlF5B4R+RKYTAxbMB6Ph48//phf/OIXfPzxxwwZMiRWlzbG1JLE68QQJ+ciBV53C5ChqndGuM7VwNUA3bp1G7Jt27Ya711aWkpaWlqd4jbG1ExE1qhqbo2vc/tIIhHpDixR1b41vTY3N1dXr16dgKiMMdE4TTBuzSL1Cnl6AbDJjTiMMfHl1jqY+0SkN1ABbAOucSkOY0wcuXUu0o/duK8xJrGa1FYBY0xiWYIxxsSN67NItSEiu/GP2dSkLbAnzuE41VBiaShxQMOJxeKozmks3VW1XU0vSqoE45SIrHYyhZYIDSWWhhIHNJxYLI7qYh2LdZGMMXFjCcYYEzeNNcHMcTuAEA0lloYSBzScWCyO6mIaS6McgzHGNAyNtQVjjGkALMEYY+Km0SaYRJXldBDHbBHZFIjlFRHJciOOQCwXisgGEakQkYRPi4rIaBHZLCKficiMRN8/JI4nRWSXiHziVgyBOLqKyDsisjHw/3KDS3FkiMhHIrI+EMesmF1cVRvlG9Aq5PH1wF9diuNHQGrg8f3A/S5+T74H9AbeBXITfG8P8DlwEpAGrAdOdun78ENgMPCJW/8XgTg6AYMDj1sC/3HjewII0CLw2AusBIbG4tqNtgWjcSzLWcs4lqpqWeDpCqCLG3EEYtmoqptduv2pwGequkVVS4EXgLFuBKKq/wL2uXHvKnF8raofBx5/C2wEclyIQ1X1UOCpN/AWk9+XRptgIH5lOevhSuA1t4NwSQ7wZcjzHbjwy9RQiUgPYBD+1oMb9/eIyDpgF/CmqsYkjqROMCKyTEQ+CfM2FkBVb1PVrsBzwC/diiPwmtuAskAsceMkFpeEOyvG1kgAItICWAD8qkrLO2FUtVz9p3x0AU4VkRorTDqR1AevqerZDl/6PLAECFv3N95xiMgVwHnAWRro6MZLLb4nibYD6BryvAvwlUuxNBgi4sWfXJ5T1YVux6OqRSLyLjAaqPcgeFK3YKJpKGU5RWQ0cDNwgaoecSOGBmIV0EtEThSRNOBi4FWXY3KV+E8AfALYqKoPuRhHu+DspohkAmcTo9+XRruSV0QW4J8xOVaWU1ULXYjjMyAd2Bt41wpVdaVEqIiMB/4EtAOKgHWqOiqB9z8XeAT/jNKTqnpPou5dJY5/AMPxlybYCdypqk+4EMf/Au8DBfh/TgFuVdV/JjiO/sDT+P9fUoAXVfW3Mbl2Y00wxhj3NdoukjHGfZZgjDFxYwnGGBM3lmCMMXFjCcYYEzeWYAwAItImsPN8nYh8IyKFIc/TXIzrbBHJc+v+pn6SeiWviR1V3QsMBBCRu4BDqvpg6GsCC8NEVSuqX8GY6qwFY6ISkZ6BvUx/BT4GuopIUcjHLxaRuYHHHURkoYisDtQXGRrmeqsD55IHny8XkQEiMlREPhSRtSLyQZWV2MHX3i0ivwp5vklEugQeXxG45zoR+YuI2M92A2D/CcaJk4EnVHUQEG019B+BB9R/rs5FwNwwr5kf+BiB5NBGVdfjL1Xwv4F7/A6422lwgY1544EfBDbspeLfimBcZl0k48TnqrrKwevOBnr7e1IAZItIpqoWh7zmRWAR/iQyKfAcIAt4RkS+U4f4zgZOAVYH7p1J5dIQxiWWYIwTh0MeV1C59EJGyGMBTg0UlApLVbeJyCERORl/gpkS+NA9wBuq+hcR6Qm8HubTy6jc6g7eW/DvbZrp5IsxiWNdJFMrgQHe/SLSKzDOMT7kw8uAa4NPRGRghMvMB24B0lX108D7WnO8+zUlwud9AQwJXPtUjpd/WAZcJCJtAx9rIyLdavFlmTixBGPq4mb8LYy38Nd5CboWOF38Bc4/BaZG+PyXgEs53j0Cf73i2SLyQZT7vgR0EJG1wFXAFgBVLQBmActEJB9YCnSo9VdlYs52Uxtj4sZaMMaYuLEEY4yJG0swxpi4sQRjjIkbSzDGmLixBGOMiRtLMMaYuPn/6gJf7hgLy1kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb7bacc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 还可以观察预测值和真值的散点图\n",
    "plt.figure(figsize=(4,3))\n",
    "plt.scatter(y_train,y_train_pred_lr)\n",
    "plt.plot([-3,3],[-3,3],'--k') # 数据已归一化，?????\n",
    "plt.axis('tight')\n",
    "plt.xlabel('True value')\n",
    "plt.ylabel('Predicted value')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结论：？？？？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.2、正则化的线性回归（L2正则-->岭回归）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of RidgeCV on test is 0.672073075409118\n",
      "The r2 score of RidgeCV on train is 0.7540516391069173\n"
     ]
    }
   ],
   "source": [
    "# 岭回归/L2正则\n",
    "from sklearn.linear_model import RidgeCV\n",
    "\n",
    "# 设置超参数（正则参数）范围\n",
    "alphas = [0.01,0.1,1,10,100]\n",
    "\n",
    "# 生成一个RidgeCV实例\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True)\n",
    "\n",
    "#模型训练\n",
    "ridge.fit(X_train,y_train)\n",
    "\n",
    "# 预测\n",
    "y_test_pred_ridge = ridge.predict(X_test)\n",
    "y_train_pred_ridge = ridge.predict(X_train)\n",
    "\n",
    "# 评估，使用r2_score评价模型在测试集合训练集上的性能\n",
    "print 'The r2 score of RidgeCV on test is', r2_score(y_test,y_test_pred_ridge)\n",
    "print 'The r2 score of RidgeCV on train is', r2_score(y_train,y_train_pred_ridge)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAH45JREFUeJzt3X1wFPed5/H3VxIPNmBjQGDMMzEGk/Bo4XXixI4xSfkhwfgBiHdzm+S8581dpWr3UtmLU75y6nK1dRX7blOV2+xuHjab5DaVSMLGjxDbwdjOZkOs4RmMsTF+mAGBhMGAwAgkfe+P7jHDuCVGHvU8fl5VKs10/7r72y3QR9098x1zd0RERD6smmIXICIi5U1BIiIieVGQiIhIXhQkIiKSFwWJiIjkRUEiIiJ5UZCIiEheFCQiIpIXBYmIiOSlrtgFFMKYMWN86tSpxS5DRKSsbNy48ZC7159vXFUEydSpU0kkEsUuQ0SkrJjZW7mM06UtERHJi4JERETyoiAREZG8KEhERCQvChIREcmLgkRERPKiIBERkbwoSEREKlDqyEm++5tXaDt+KvZtKUhERCpQcyLFP73wOme6PfZtKUhERCpMT4+zamOKT14+hgkjL4h9ewoSEZEK8/vXD7Hv3fdY0TCpINtTkIiIVJimRIqRFw7isx8dV5DtKUhERCrIuydP8/TOAyybP4EhdbUF2aaCRESkgjy2ZT+nu3oKdlkLFCQiIhWlsSXJxyZcxOzLLirYNhUkIiIVYse+o7zceoyVBTwbAQWJiEjFaEokGVxXw9J5Ewq6XQWJiEgFOHWmm0c37+Pmj13KxRcOKui2FSQiIhXg6Z0HOHaqq6A32dMUJCIiFaApkWTiJRfw8emjC75tBYmISJlLHj7J7/e8w/KrJlFTYwXfvoJERKTMNW9MYQZ3NUwsyvYVJCIiZay7x1mVSPKpGfUFadAYJdYgMbObzGy3me0xs/si5n/dzF42s21mts7MpmTMe9DMdprZLjP7vplZOP0qM9servP96SIi1ej3ew6x/+gpVhTpbARiDBIzqwV+ANwMzAbuNrPZWcM2Aw3uPhdYBTwYLvsJ4FpgLvAxYBFwfbjMPwL3AjPCr5vi2gcRkVLXlEgy8sJBfGZ2YRo0RonzjORqYI+773X308CvgdsyB7j7enc/GT7dAKQj1YGhwGBgCDAIOGhm44GL3P0P7u7AL4BlMe6DiEjJOnLiNM/sPFjQBo1R4gySCUAy43kqnNabe4C1AO7+B2A90Bp+Pe3uu8LlU/1Yp4hIxXpsyz5Odxe2QWOUuhjXHXXvIvIzH83si0AD4eUrM7scuJKzZyjPmtl1wHv9WOe9BJfAmDx5cr8KFxEpde5OYyLFnAkXF7RBY5Q4z0hSQGZMTgT2Zw8ysyXA/cBSd+8MJ98ObHD3DnfvIDhTuSZcZ+Ydpch1Arj7j9y9wd0b6uvr894ZEZFSsmPfMXa1HmPFouKejUC8QdICzDCzaWY2GPgC8HjmADNbAPyQIETaMma9DVxvZnVmNojgTGWXu7cCx83smvDVWn8OPBbjPoiIlKSmRJIhdTUsnXdZsUuJL0jcvQv4GvA0sAtocvedZvYdM1saDnsIGA40m9kWM0sHzSrgdWA7sBXY6u5PhPP+M/ATYE84Zm1c+yAiUopOnenm0S1hg8YLCtugMUqc90hw9zXAmqxpD2Q8XtLLct3AX/YyL0HwkmARkar09M4DHC9Sg8Yoeme7iEiZaWxJMmnUBVxThAaNURQkIiJlJHn4JP/+evEaNEZRkIiIlJHmRDJo0HhV8VqiZFOQiIiUie4eZ9XGFNfNqOeyIjVojKIgEREpE//2foPG0rjJnqYgEREpE02JJJdcOIgls8cWu5RzKEhERMrAkROneXbnQZYtKG6DxigKEhGRMrB6c9CgcWUJtETJpiARESlx7k5TIsnciRcz69LiNmiMoiARESlx2/cd5ZUDx0vuJnuagkREpMSlGzR+vgQaNEZRkIiIlLBTZ7p5bMt+bpkzviQaNEZRkIiIlLDf7AgaNC5vKJ13smdTkIiIlLDGliSTR13INdNKo0FjFAWJiEiJevudk/xh7zusaJhYMg0aoyhIRERKVPPGJDUGd5ZQg8YoChIRkRL0foPGK+oZf3HpNGiMoiARESlBv3utndYSbNAYRUEiIlKCmhJJRg0bzJIrxxW7lPNSkIiIlJjDJ07z7MsHWTZ/AoPrSv/XdOlXKCJSZVZv3seZbi/JBo1RFCQiIiXE3WlOJJk38WJmXjqi2OXkREEiIlJCtqXCBo1lcjYCChIRkZLSlEgydFDpNmiMoiARESkR753u5vEt+7nlY+O5aGhpNmiMoiARESkRv9nZyvHOLpaXwXtHMilIRERKRGNLkimjL+Sa6aOKXUq/KEhERErAW++cYMPew6xomIRZ6TZojKIgEREpAc2JVNCgcWFpN2iMoiARESmydIPG66+o59KLhxa7nH5TkIiIFNmLr7Vz4Fh5NGiMoiARESmyppagQeONZdCgMYqCRESkiN7p6OS3uw5y+4LyaNAYpTyrFhGpEOkGjeV6WQsUJCIiRePuNCWSzJs0smwaNEZRkIiIFMnW1FFePdjByjI+GwEFiYhI0aQbNH5u3vhil5IXBYmISBG8d7qbJ7bs55Y55dWgMYqCRESkCNbuCBo0lvNN9rRYg8TMbjKz3Wa2x8zui5j/dTN72cy2mdk6M5sSTr/BzLZkfJ0ys2XhvJ+Z2RsZ8+bHuQ8iInFobEkydfSF/Mm08mrQGCW2IDGzWuAHwM3AbOBuM5udNWwz0ODuc4FVwIMA7r7e3ee7+3xgMXASeCZjub9Jz3f3LXHtg4hIHN48dII/vnGY5WXYoDFKnGckVwN73H2vu58Gfg3cljkgDIyT4dMNQFS3sruAtRnjRETKWvPGZNk2aIwSZ5BMAJIZz1PhtN7cA6yNmP4F4FdZ0/42vBz2PTMbkl+ZIiKF09Xdw6qNKT49c2xZNmiMEmeQRJ2veeRAsy8CDcBDWdPHA3OApzMmfwuYBSwCRgHf7GWd95pZwswS7e3t/a9eRCQGv3vtEAePdbKioTLORiDeIEkBmS9HmAjszx5kZkuA+4Gl7t6ZNXsFsNrdz6QnuHurBzqBfyG4hPYB7v4jd29w94b6+vo8d0VEZGA0tiQZPWwwi2eVZ4PGKHEGSQsww8ymmdlggktUj2cOMLMFwA8JQqQtYh13k3VZKzxLwYI7VMuAHTHULiIy4CqhQWOUurhW7O5dZvY1gstStcBP3X2nmX0HSLj74wSXsoYDzeErF95296UAZjaV4IzmhaxV/9LM6gkunW0BvhrXPoiIDKTVm/fR1eOsWFT+7x3JFFuQALj7GmBN1rQHMh4v6WPZN4m4Oe/uiwewRBGRgnB3GluSzJ80kivGlW+DxiiVc24lIlLCtiTf5bW2DlZW2NkIKEhERAqiKZHigkG1fG5ueTdojKIgERGJ2cnTXTyxNWjQOKLMGzRGUZCIiMRs7fYDdHR2VeRlLVCQiIjErjGRZNqYYSyaekmxS4mFgkREJEZvHDrBS28cZnnDxIpo0BhFQSIiEqPmRGU1aIyiIBERiUm6QeMNM8cy7qLKaNAYRUEiIhKTF19rp+14J8sr4FMQ+6IgERGJSWNLkjHDB3PjlWOLXUqsFCQiIjE41NHJul1t3L5gAoNqK/tXbWXvnYhIkazeFDZorPDLWqAgEREZcO5OUyLJgskjmVFhDRqjKEhERAbY5nSDxio4GwEFiYjIgGtOJLlgUC23VmCDxigKEhGRARQ0aGzl1rmV2aAxioJERGQAPbWttaIbNEZRkIiIDKDmRIrpY4bRMKUyGzRGUZCIiAyQve0dvPTmYZY3TKrYBo1Rcg4SM/ukmX0lfFxvZtPiK0tEpPw0b0xRW2PcuXBCsUspqJyCxMy+DXwT+FY4aRDwr3EVJSJSbrq6e3h4Y4obZtYztoIbNEbJ9YzkdmApcALA3fcDlf8uGxGRHL3wanU0aIySa5CcdncHHMDMhsVXkohI+Uk3aFw8q7IbNEbJNUiazOyHwEgz+0/Ab4Efx1eWiEj5aD/eyXOvtHHHwokV36AxSl0ug9z9f5vZZ4BjwEzgAXd/NtbKRETKxOrNqbBBY+V+CmJfcgqS8FLWc+7+rJnNBGaa2SB3PxNveSIipS1o0Jhi4eSRXD62Om8d53oO9iIwxMwmEFzW+grws7iKEhEpF5vefpc9bR1V9U72bLkGibn7SeAO4P+6++3A7PjKEhEpD00tSS4cXMutcy8rdilFk3OQmNnHgT8Dngqn5XRZTESkUp3o7OLJbfu5dc54hg+p3l+JuQbJXwH3AY+4+87wXe3PxVeWiEjpe2p7KydOd1f1ZS3I/aziJNAD3G1mXwSM8D0lIiLVqjmRZHr9MK6qogaNUXINkl8C3wB2EASKiEhVe729g5Y3j3DfzbOqqkFjlFyDpN3dn4i1EhGRMtKcCBo03lFlDRqj5Bok3zaznwDrgM70RHd/JJaqRERKWFd3Dw9vSnHDzLGMHVFdDRqj5BokXwFmEXT9TV/ackBBIiJV5/nd7bQf76zad7JnyzVI5rn7nFgrEREpE42JJGOGD+GGKmzQGCXXl/9uMDO9AVFEql7b8VM890obdy6cUJUNGqPkekbySeBLZvYGwT0SA9zd58ZWmYhICVq9aR/dPV6VnzvSm1yD5KZYqxARKQPuTmMiyVVTLuHyscOLXU7JyLWN/FtxFyIiUuo2vX2Eve0nePDOjxS7lJIS6wU+M7vJzHab2R4zuy9i/tfN7GUz22Zm68xsSjj9BjPbkvF1ysyWhfOmmdkfzew1M2s0s8Fx7oOISFrj+w0axxe7lJISW5CYWS3wA+Bmgk7Bd0fcsN8MNIT3WlYBDwK4+3p3n+/u84HFBC1angmX+S7wPXefARwB7olrH0RE0oIGja18bu54hlVxg8YocZ6RXA3scfe97n4a+DVwW+aAMDBOhk83AFEvyr4LWOvuJy3oQ7CYIHQAfg4si6V6EZEMT21r5aQaNEaKM0gmAMmM56lwWm/uAdZGTP8C8Kvw8WjgXXfvOt86zexeM0uYWaK9vb1fhYuIZGsKGzQunFzdDRqjxBkkUV3MIjsGhx2FG4CHsqaPB+YAT/d3ne7+I3dvcPeG+vr6nIsWEcm2p62DxFtHWNkwqeobNEaJ80JfCsg8B5wI7M8eZGZLgPuB6929M2v2CmB1xmfDHwJGmlldeFYSuU4RkYHUvDFJbY1xuxo0RorzjKQFmBG+ymowwSWqxzMHmNkC4IfAUndvi1jH3Zy9rIW7O7Ce4L4JwJeAx2KoXUQEgDPdPTy8cR+LZ6lBY29iC5LwjOFrBJeldgFN4acrfsfMlobDHgKGA83hy3zfDxozm0pwRvNC1qq/CXzdzPYQ3DP557j2QUTk+d3tHOroZIXeyd6rWF/D5u5rgDVZ0x7IeLykj2XfJOJGurvvJXhFmIhI7BpbktSPGMINM3WvtTfqOCYi0ou2Y6dYv7uNOxZOoE4NGnulIyMi0otHNgcNGnVZq28KEhGRCO5OU0uShimX8JF6NWjsi4JERCTCxreOsPfQCVboneznpSAREYnQ2JJk2OBabp2jBo3noyAREcnS0dnFU9tb+dzcy9SgMQcKEhGRLE9t28/J0926rJUjBYmISJamRIqP1A9j4eSRxS6lLChIREQy7Gk7zsa3jrBykRo05kpBIiKSoTmRoq7GuH1B1McjSRQFiYhI6Ex3Dw9vSrF41ljqRwwpdjllQ0EiIhJa/0obhzpO653s/aQgEREJNSWCBo2fVoPGflGQiIiQbtDYzp0LJ6pBYz/paImIAA9vSjdo1E32/lKQiEjVc3eaE0munjqK6WrQ2G8KEhGpeomwQeNynY18KAoSEal67zdonKsGjR+GgkREqlpHZxdPbWvl8/Mu48LBatD4YShIRKSqPbl1P++dUYPGfChIRKSqNSWSXD52OAsmqUHjh6UgEZGqtaftOJvefpeVDWrQmA8FiYhUraZ0g8aFE4pdSllTkIhIVTrT3cMjm1LceOVYxgxXg8Z8KEhEpCqt26UGjQNFQSIiVak5kWTsiCFcf4UaNOZLQSIiVefgsVOs393GnVepQeNA0BEUkarz8KYUPY4uaw0QBYmIVJWgQWOKq6eNYtqYYcUupyIoSESkqrS8eYQ3Dp3Q2cgAUpCISFVpbEkyfEgdt8y5tNilVAwFiYhUjeOnzrBmeyufnzdeDRoHkIJERKrGk9tagwaNuqw1oBQkIlI1mhJJZowdznw1aBxQChIRqQqvHTzO5rffZeUiNWgcaAoSEakKjS1J6mqMZQvUoHGgKUhEpOKd7uph9eZ9LLlynBo0xkBBIiIV77lXDvLOidOsWDSx2KVUJAWJiFS8pkSKcRcN4boZatAYh1iDxMxuMrPdZrbHzO6LmP91M3vZzLaZ2Tozm5Ixb7KZPWNmu8IxU8PpPzOzN8xsS/g1P859EJHyduDoKZ7f3cadC9WgMS6xHVUzqwV+ANwMzAbuNrPZWcM2Aw3uPhdYBTyYMe8XwEPufiVwNdCWMe9v3H1++LUlrn0QkfKnBo3xizOerwb2uPtedz8N/Bq4LXOAu69395Ph0w3ARIAwcOrc/dlwXEfGOBGRnAQNGpP8ybRRTFWDxtjEGSQTgGTG81Q4rTf3AGvDx1cA75rZI2a22cweCs9w0v42vBz2PTOLfAmGmd1rZgkzS7S3t+ezHyJSpl564zBvvnNSZyMxizNIot7x45EDzb4INAAPhZPqgE8B3wAWAdOBL4fzvgXMCqePAr4ZtU53/5G7N7h7Q329brCJVKPGRLpB4/hil1LR4gySFJD5Z8BEYH/2IDNbAtwPLHX3zoxlN4eXxbqAR4GFAO7e6oFO4F8ILqGJiJzjbIPGy7hgcO35F5APLc4gaQFmmNk0MxsMfAF4PHOAmS0AfkgQIm1Zy15iZulTicXAy+Ey48PvBiwDdsS4DyJSpp7Y2sqpMz2sXKTLWnGLrY+yu3eZ2deAp4Fa4KfuvtPMvgMk3P1xgktZw4HmsPfN2+6+1N27zewbwLowMDYCPw5X/cswYAzYAnw1rn0QkfLVmEhyxbjhzJt4cbFLqXixNuR39zXAmqxpD2Q8XtLHss8CcyOmLx7IGkWk8uw+cJytyXf577deqQaNBaB354hIxWlKJBlUa9yuBo0FoSARkYqS2aBxtBo0FoSCREQqyrpdBzl84rTeO1JAChIRqShNiSSXXjSU667Q+8cKRUEiIhXjwNFTvPBqO3ddNZHaGt1kLxQFiYhUjHSDxuUN+tyRQlKQiEhF6OlxmhJJrpk+iimj1aCxkBQkIlIRXnrzMG+pQWNRKEhEpCI0tSQZMaSOmz+mBo2FpiARkbJ37NQZ1uxo5fPz1aCxGBQkIlL2nti6P2jQqMtaRaEgEZGy19SSZOa4EcxVg8aiUJCISFl75cAxtqaOsmLRJDVoLBIFiYiUtaaWlBo0FpmCRETKVtCgMcVnZo9j1LDBxS6nailIRKRs/XbXQY6cPMNy3WQvKgWJiJStpkSS8RcP5boZatBYTAoSESlLrUff40U1aCwJChIRKUsPbwwbNF6ly1rFpiARkbITNGhM8fHpo5k8+sJil1P1FCQiUnY2vPEObx8+yYpFahdfChQkIlJ2mhMpRgxVg8ZSoSARkbJy9L0zrNneytJ5lzF0kBo0lgIFiYiUlSe27qezq4eVi3STvVQoSESkrDQlksy6dARzJqhBY6lQkIhI2djVeoxtqaOsaFCDxlKiIBGRstGUSDKo1limBo0lpa7YBZSyv/x/CV589dAHpvf2h1DU5N7+aoqcOhDrjZjc299tUevofWxulZkFU2vMqLFgG2bB9GCaYelx4ZiacOU14dia95cJ10HUNKOmJvieuczZ9ae3f+6ykbVxtg7LmF6Ttb6z+/DBZc9ux86pt8bOXTZqXzLXWWPn1lJjwX6m153T+HPmQ01NP8enf07hsrW9za/hvOsbSJ1d3Ty6eR+fnX2pGjSWGAVJHxbPGsuU0cPOmebukWOjJkeP7G1s7uvtTVRtcdXQ+3od92CZHnd6PFhvMC143uPBltJj3x8XjnEyp4XL4vT0hN8d3Hvw7rPryhzbk/X97DLp5+n1n631/LX1sWzG+nv68fOqBh8IpuzgqTlPkGVMO9PTEzZo1HtHSo2CpA8rF00udglShs4JNDgndM6Z1nPu856s8OrpyXx+blhFjnc/O7+nn+PPWX8Q2pnbPWd+T+byOa4vc3xPP8dnPF9y5Tg+pQaNJUdBIjLA3r+U1euFQpHKopvtIiKSFwWJiIjkRUEiIiJ5UZCIiEheFCQiIpIXBYmIiORFQSIiInlRkIiISF6st5YflcTM2oG3PuTiY4APNtwqPtXVP6qrf1RX/1RqXVPc/bytBKoiSPJhZgl3byh2HdlUV/+orv5RXf1T7XXp0paIiORFQSIiInlRkJzfj4pdQC9UV/+orv5RXf1T1XXpHomIiORFZyQiIpIXBUkWM3vIzF4xs21mttrMRvYy7iYz221me8zsvgLUtdzMdppZj5n1+ioMM3vTzLab2RYzS5RQXYU+XqPM7Fkzey38fkkv47rDY7XFzB6PsZ4+99/MhphZYzj/j2Y2Na5a+lnXl82sPeMY/UWB6vqpmbWZ2Y5e5puZfT+se5uZLSyBmj5tZkczjtUDcdcUbneSma03s13h/8W/ihgT7/Hy8FPK9OXpj6r9LFAXPv4u8N2IMbXA68B0YDCwFZgdc11XAjOB54GGPsa9CYwp4PE6b11FOl4PAveFj++L+jmG8zoKcIzOu//AfwH+KXz8BaCxROr6MvD3hfr3lLHd64CFwI5e5t8CrAUMuAb4YwnU9GngySIcq/HAwvDxCODViJ9jrMdLZyRZ3P0Zd+8Kn24Aoj4g+mpgj7vvdffTwK+B22Kua5e7745zGx9GjnUV/HiF6/95+PjnwLKYt9eXXPY/s95VwI1mFvdHLBbj55ITd38RONzHkNuAX3hgAzDSzMYXuaaicPdWd98UPj4O7AImZA2L9XgpSPr2HwlSPNsEIJnxPMUHf3DF4sAzZrbRzO4tdjGhYhyvce7eCsF/NGBsL+OGmlnCzDaYWVxhk8v+vz8m/EPmKDA6pnr6UxfAneHlkFVmNinmmnJVqv8HP25mW81srZl9tNAbDy+JLgD+mDUr1uNVlZ/Zbma/BS6NmHW/uz8Wjrkf6AJ+GbWKiGl5v/wtl7pycK277zezscCzZvZK+JdUMesq+PHqx2omh8drOvCcmW1399fzrS1LLvsfyzE6j1y2+QTwK3fvNLOvEpw1LY65rlwU43idzyaCliIdZnYL8Cgwo1AbN7PhwMPAX7v7sezZEYsM2PGqyiBx9yV9zTezLwGfA2708AJjlhSQ+ZfZRGB/3HXluI794fc2M1tNcPkiryAZgLoKfrzM7KCZjXf31vAUvq2XdaSP114ze57gr7mBDpJc9j89JmVmdcDFxH8Z5bx1ufs7GU9/THDfsBTE8m8qH5m/vN19jZn9g5mNcffYe3CZ2SCCEPmluz8SMSTW46VLW1nM7Cbgm8BSdz/Zy7AWYIaZTTOzwQQ3R2N7xU+uzGyYmY1IPyZ44UDkK0wKrBjH63HgS+HjLwEfOHMys0vMbEj4eAxwLfByDLXksv+Z9d4FPNfLHzEFrSvrOvpSguvvpeBx4M/DVyNdAxxNX8osFjO7NH1fy8yuJvj9+k7fSw3Idg34Z2CXu/9dL8PiPV6FfoVBqX8BewiuJW4Jv9KvpLkMWJMx7haCV0e8TnCJJ+66bif4q6ITOAg8nV0XwatvtoZfO0ulriIdr9HAOuC18PuocHoD8JPw8SeA7eHx2g7cE2M9H9h/4DsEf7AADAWaw39/LwHT4z5GOdb1v8J/S1uB9cCsAtX1K6AVOBP++7oH+Crw1XC+AT8I695OH69kLGBNX8s4VhuATxToWH2S4DLVtozfW7cU8njpne0iIpIXXdoSEZG8KEhERCQvChIREcmLgkRERPKiIBERkbwoSET6YGYdeS6/KnzXfF9jnrc+OifnOiZrfL2Z/SbX8SL5UJCIxCTstVTr7nsLvW13bwdazezaQm9bqo+CRCQH4TuCHzKzHRZ83svKcHpN2Apjp5k9aWZrzOyucLE/I+Md9Wb2j2GDyJ1m9j962U6Hmf0fM9tkZuvMrD5j9nIze8nMXjWzT4Xjp5rZ78Lxm8zsExnjHw1rEImVgkQkN3cA84F5wBLgobB9yB3AVGAO8BfAxzOWuRbYmPH8fndvAOYC15vZ3IjtDAM2uftC4AXg2xnz6tz9auCvM6a3AZ8Jx68Evp8xPgF8qv+7KtI/Vdm0UeRD+CRBF9xu4KCZvQAsCqc3u3sPcMDM1mcsMx5oz3i+ImztXxfOm03Q1iJTD9AYPv5XILMBX/rxRoLwAhgE/L2ZzQe6gSsyxrcRtKoRiZWCRCQ3vX3IVF8fPvUeQQ8tzGwa8A1gkbsfMbOfpeedR2YPo87wezdn/+/+V4IeZ/MIrjCcyhg/NKxBJFa6tCWSmxeBlWZWG963uI6gueK/EXzwU42ZjSP4uNW0XcDl4eOLgBPA0XDczb1sp4ag+y/An4br78vFQGt4RvQfCD4+N+0KSqP7s1Q4nZGI5GY1wf2PrQRnCf/N3Q+Y2cPAjQS/sF8l+GS6o+EyTxEEy2/dfauZbSboDrsX+H0v2zkBfNTMNobrWXmeuv4BeNjMlhN05z2RMe+GsAaRWKn7r0iezGy4B5+KN5rgLOXaMGQuIPjlfm14byWXdXW4+/ABqutF4DZ3PzIQ6xPpjc5IRPL3pJmNBAYD/9PdDwC4+3tm9m2Cz8Z+u5AFhZff/k4hIoWgMxIREcmLbraLiEheFCQiIpIXBYmIiORFQSIiInlRkIiISF4UJCIikpf/D+V4qz4FfDcRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xc4a8a58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('alpha is', 10.0)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.635529</td>\n",
       "      <td>0.619655</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.297763</td>\n",
       "      <td>0.280457</td>\n",
       "      <td>season</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.007874</td>\n",
       "      <td>0.008185</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.001075</td>\n",
       "      <td>0.017236</td>\n",
       "      <td>mnth</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.033981</td>\n",
       "      <td>-0.033886</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.071831</td>\n",
       "      <td>-0.068944</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-0.111401</td>\n",
       "      <td>-0.110003</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.214388</td>\n",
       "      <td>-0.211558</td>\n",
       "      <td>weathersit</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    coef_lr  coef_ridge     columns\n",
       "6  0.635529    0.619655       atemp\n",
       "0  0.297763    0.280457      season\n",
       "4  0.007874    0.008185  workingday\n",
       "1  0.000000    0.000000          yr\n",
       "2 -0.001075    0.017236        mnth\n",
       "3 -0.033981   -0.033886     holiday\n",
       "7 -0.071831   -0.068944         hum\n",
       "8 -0.111401   -0.110003   windspeed\n",
       "5 -0.214388   -0.211558  weathersit"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse_mean = np.mean(ridge.cv_values_,axis=0)\n",
    "plt.plot(np.log10(alphas),mse_mean.reshape(len(alphas),1))\n",
    "\n",
    "#这是为了标出最佳参数的位置，不是必须\n",
    "#plt.plot(np.log10(ridge.alpha_)*np.ones(3),[0.28,0.29,0.30])\n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()\n",
    "\n",
    "print('alpha is',ridge.alpha_)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns),\"coef_lr\":list((lr.coef_.T)),\"coef_ridge\":list(ridge.coef_.T)})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.3正则化的线性回归（L1正则-->Lasso）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r2 score of LassoCV on test is 0.611961704945087\n",
      "The r2 score of LassoCV on train is 0.6845451785239587\n"
     ]
    }
   ],
   "source": [
    "### Lasso / L1正则\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "\n",
    "# 设置超参数搜索范围\n",
    "# alphas=[0.01,0.1,1,10,100]\n",
    "\n",
    "# 生成一个LassoCV实例\n",
    "# lasso = LassoCV(alphas=alphas)\n",
    "\n",
    "lasso = LassoCV()\n",
    "\n",
    "#训练（内含CV）\n",
    "lasso.fit(X_train,y_train)\n",
    "\n",
    "#测试\n",
    "y_test_pred_lasso = lasso.predict(X_test)\n",
    "y_train_pred_lasso = lasso.predict(X_train)\n",
    "\n",
    "# 评估，使用r2_score评价模型在测试集和训练集上的性能\n",
    "print 'The r2 score of LassoCV on test is', r2_score(y_test,y_test_pred_lasso)\n",
    "print 'The r2 score of LassoCV on train is', r2_score(y_train,y_train_pred_lasso)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xl4lNXd//H3N/u+h50AYZUdjKCCSqu11lat1g21tYpSq120y9P2aatP7a97aze1VimPWqvVqo9rtXWnVUEBWcIWCPuWhCSELISQ5Pz+mMEmmmWA3LlnMp/XdeVikjkz870zJJ/c55z7HHPOISIickSM3wWIiEh4UTCIiEg7CgYREWlHwSAiIu0oGEREpB0Fg4iItONZMJjZQjMrN7PiTu7/ppmtCH4Um1mLmeV4VY+IiITGvLqOwcxOB+qAB51zE7tpex5wi3Puo54UIyIiIfPsjME5twioCrH5XOARr2oREZHQxfldgJmlAOcAXwqlfV5enhs+fLinNYmI9DXLli3b55zLD6Wt78EAnAe86Zzr9OzCzOYD8wEKCgpYunRpb9UmItInmNm2UNuGw6yky+mmG8k5d69zrsg5V5SfH1LgiYjIMfI1GMwsEzgDeNrPOkRE5D8860oys0eAOUCeme0EbgPiAZxz9wSbXQj80zlX71UdIiJydDwLBufc3BDa3A/c71UNIiJy9MJhjEFERMKIgkFERNpRMIiISDsKBhGRMFdz8DD3LiplyebKXnm9cLjATUREOrCjqoGFb27hsXd3UN/UwhfnjGRmYa7nr6tgEBEJM6UVddz16iaeWrGLGDPOnzKIa2ePYOLgzF55fQWDiEgYcM6xfPt+7n9rK8+t2k1SXCzzZo9g3uxCBmQm9WotCgYRER/tb2ji+dV7eGjxdtbtOUBaYhxfOH0k1502gry0RF9qUjCIiPSy6vomXi8p57mVe1i0sYLDLY5xA9L50YUT+fTUwaQm+vurWcEgIuKxg00trNixnyVbKnmjpIKVO/bT6mBQZhKfP3U4n5o8iMlDMjEzv0sFFAwiIj2qoamZTeV1rNl9gDW7a1i9s4Y1uw/Q3OowgylDsvjyR0dzxth8pg7JIiYmPMKgLQWDiMgxqKw7xKKNFeyqPsiu/QfZXtXA5op69tQ0vt8mPTGO8YMymH96IUXDs5lekE1WSoKPVYdGwSAicgxue2YNz63aA0BuagJDclI4pTCXwvxURvVLY/zATIbmJIdN99DRUDCIiBwl5xzvbKniExMHcMelU0lOiPW7pB6lJTFERI7SzuqDlNce4tSRuX0uFEDBICJy1JZvrwbgxGE5PlfiDQWDiMhRWrq1mtSEWMYOSPe7FE94FgxmttDMys2suIs2c8xshZmtMbM3vKpFRKQnLdtWzbSCbGLDcKppT/DyjOF+4JzO7jSzLOBu4Hzn3ATgEg9rERHpEXWHmlm/9wDTh2X7XYpnPAsG59wioKqLJlcATzrntgfbl3tVi4hITzly1XKRgsETY4BsM3vdzJaZ2ed8rEVEJCRLt1ZjBlMLsvwuxTN+XscQB5wInAkkA2+b2WLnXMkHG5rZfGA+QEFBQa8WKSLS1rLt1Yztn05GUrzfpXjGzzOGncCLzrl659w+YBEwpaOGzrl7nXNFzrmi/Pz8Xi1SROSI1lbHe9uq+/T4AvgbDE8Dp5lZnJmlADOBdT7WIyLSpY3lddQeau7T4wvgYVeSmT0CzAHyzGwncBsQD+Ccu8c5t87MXgRWAa3AAudcp1NbRUT8tnRbYD7NiQqGY+OcmxtCm18Av/CqBhGRnrRsWzV5aQkU5KT4XYqndOWziEgInHMsLq3kpOE5Ebli6tFQMIiIhGBrZQO7axo5dVSe36V4TsEgIhKCt0r3ATBrZK7PlXhPwSAiEoK3NlUyMDOJEXmpfpfiOQWDiEg3Wlsdb5Xu45SRuX1+fAEUDCIi3Vq39wDVDYeZNbLvjy+AgkFEpFtvbaoEYFYUDDyDgkFEpFtvlu6jMD+VAZlJfpfSKxQMIiJdaGpu5Z0tVVHTjQQKBhGRLq3cuZ+GphZmjer701SPUDCIiHThrU2VmMHJhQoGEREB3ty0jwmDMshKSfC7lF6jYBAR6cT+hiaWba9mzph+fpfSqxQMIiKdeH1DBS2tjjNPUDCIiAjw8roy8tISmTKk7+7v3BEFg4hIBw63tPJGSQUfHZdPTEzfXwajLQWDiEgH3t1SRW1jM2ee0N/vUnqdl1t7LgQ+BZQ75yZ2cP8cAvs+bwl+6Unn3O1e1fPahnJufbqYtMR40hPjSEuKIz34kZEUT3pSPBnJcaQnxZOVHE9WSjxZyQlkpgTaR9tfDCLR7uV15STExXDa6Oi5sO0Iz4IBuB+4E3iwizb/cs59ysMa3peVHE/RsBxqG5upO3SYsgONbCpvprbxMLWNzTS3uk4fG2OQkRxPdkoC2Snx5KQmBD8SyU1NIDctgby0xPf/zUlNID5WJ2Mikco5xyvryzh1ZC4pCV7+mgxPXu75vMjMhnv1/EdrWkE20wo63sDbOUfj4VYONB7mwMHD1Bw8zP6Gw1Q3NFFzMPC16uDn1Q1N7NrfyOpdNVTVN3G4peNAyUyOJy8tgQGZSfRPT6JfRhIDM5MYkBn4d1BWMrmpCVGxhK9IpCmtqGNbZQPXnVbodym+8DsKTzGzlcBu4BvOuTV+FGFmJCfEkpwQS/+M0BfJcs5Re6iZqrom9tUdYl/dISrrm9hX20Rl/SHKDxyivLaRJVuqKK9t/FCIJMTFMDgrmUFZSQzJSmFwdjIFOSkMzUmhICeFvDQFh4gfXl5XDsCZ46JrmuoRfgbDcmCYc67OzM4FngJGd9TQzOYD8wEKCgp6r8JumBkZSfFkJMUzvJtdnVpbHVUNTeytaWTX/oPs2X+QPTWN7Nx/kF3VB3l1QzkVtYfaPSYtMY5huSkMz0ulMC+VwvxUCvPSKMxPJT0p3stDE4lqr64rZ/zADAZlJftdii98Cwbn3IE2t/9uZnebWZ5zbl8Hbe8F7gUoKirqfDAgjMXEGHlpieSlJTJxcGaHbRoPt7CzuoHtVQ1sqwx8bK2sZ82uGl4s3ktLm3GQARlJjOqXxqh+aYzpn86Y/mmMHZCuwBA5Tq2tjtW7arh8xlC/S/GNb8FgZgOAMuecM7MZBKbOVvpVTzhIio9lVL90RvVL/9B9Tc2tbK+qZ1N5PaUVdZSW17Gpoo7Hlu6goanl/XYFOSmMH5jBhEEZTBySyaTBmeSlJfbmYYhEtF37D3LwcAtj+n/45zBaeDld9RFgDpBnZjuB24B4AOfcPcDFwBfNrBk4CFzunIvIs4HekBAX02FotLY6du0/SElZLev31rJ29wHW7jnAi2v2vt9mUGYSU4ZmMWVoFlOHZjFlSBbJCbG9fQgiEWFTeR0Ao/ul+VyJf7yclTS3m/vvJDCdVY5DTIwxNDhg3fZCnNrGw6zZfYDiXTWs3FnDyh37eaE4EBZxMcaEQRmcOCyHmYU5nDQ8h5zU6Fk5UqQrG8trARilYJC+Jj0pnpMLc9utIV9V38SKHdUs3VrN0m3V/GXJNha+Gbi+cGz/dE4ZmcupI3OZWZhLZrLGKiQ6bSyrIy8tMaqW2f4gBUMUyUlN4KPj+vPRcYEzi0PNLazeWcOSLVUs3lzJX9/dzv1vbSXGYPKQLE4bncdpo/OZXpBFnC7YkyixqaIuqruRQMEQ1RLjYikankPR8Bxu+sgoDjW3sHJHDf/etI9/b6zg7tdL+f2rm8hMjueMMfmceUI/5oztp7MJ6bOcc2wqq+PC6YP9LsVXCgZ5X2JcLDNG5DBjRA5f+9gYDjQe5t8b9/Hq+nJeW1/OMyt3ExdjnDIyl7MnDOCcCQPIT9eMJ+k7yg4covZQs84Y/C5AwldGUjznThrIuZMG0trqeG/Hfv65di//XFPG958q5rani5k5IpdPTg600QC2RLr/DDxH71RVUDBIiGJijBOHZXPisGy+fc44SsrqeH71Hp5btZvvPVXM/zyzhjPG5HPBtMGcPb4/SfGaDiuRZ2NZcKpqf50xiBwVM2PsgHTGDkjnlrNGs3bPAZ5ZsZtnVu7mlfXlpCfFcd6UQVx84hCmDc3Sek8SMTaW15GVEk9ulJ/9KhjkuJgZEwZlMmFQJt86ZxyLt1Ty+NKdPLl8Jw8v2c6Y/mlcflIBF00fHNXT/yQylJYHZiRF+x8zmoMoPSYmxjh1ZB53XDaVd797Fj+5aBLJCXHc/txaZvz4Fb722AqKd9X4XaZIh5xzlJTXRv34AuiMQTySnhTP3BkFzJ1RwNrdB3jkne08sXwnTy7fRdGwbObNHsHZEwYQq53xJExU1jexv+Fw1M9IAp0xSC8YPyiDH356Iov/+0y+/6nxlNce4ot/Wc6Zv3qdhxZvo/FwS/dPIuIxDTz/h4JBek1GUjzzZo/gtW/M4e4rp5OZHM/3nipm9s9e495FpdQfava7RIlim4JTVUerK0nBIL0vNsY4d9JAnrppFo9cfzLjBqTz47+vZ9bPXuWu1zYpIMQXG8vrSEuMo3+GLtrUGIP4xixwFfUpI3NZvr2aO1/dxC/+sYH/fXMLN84ZxRUzC3Q9hPSaTeV1jNKMJEBnDBImphdks/DzJ/Hkjacypn86tz+3ljN/9QZPvbeL1lZt0yHe2xgMBlEwSJiZXpDNw9efzF+um0l2ajw3P7qC8+/6N2+XRvXmfuKxmoOHqag9pBlJQQoGCUuzRuXxzE2z+fVlU6iuP8zc+xZz41+WsaOqwe/SpA/aXBGYkVSYr2AAD4PBzBaaWbmZFXfT7iQzazGzi72qRSJTTIxx4bQhvPL1M/j6x8bw2voKzrrjDe54qURTXKVHlVbUAzAyP9XnSsKDl2cM9wPndNXAzGKBnwH/8LAOiXBJ8bF8+czRvPL1Mzh7wgB+98pGzvnNIt7ctM/v0qSP2FxRR3xsYJtc8TAYnHOLgKpumn0ZeAIo96oO6TsGZSXz+7nTeGjeTACuXLCErz26gqr6Jp8rk0hXWlFHQU4K8dqpEPBxjMHMBgMXAvf4VYNEptmj83jx5tP5ykdH8eyq3Zx1xxs8vWIXzmn2khyb0op6Rmp84X1+xuNvgG8557rtLDaz+Wa21MyWVlRU9EJpEu6S4mP52tljee7Lp1GQk8JX/7qCeQ8sZW9No9+lSYRpbmllW2U9IzUj6X1+BkMR8Fcz2wpcDNxtZp/uqKFz7l7nXJFzrig/P783a5QwN3ZAOk988VRu/dR43irdx9m/Dlz7oLMHCdWO6oMcbnEU5mng+QjfgsE5N8I5N9w5Nxx4HLjROfeUX/VI5IqNMa6dPYIXvno6o/qlcfOjK7jxL8up1tiDhKC0PDBVVWcM/+HldNVHgLeBsWa208zmmdkNZnaDV68p0W1EXip/u+FU/uucsby8roxP/PZfujBOulUavIZhZJ6C4QjP1kpyzs09iraf96oOiS6xMcaNc0Zx2qh8vvLX97hiwWJumjOKm88aTZxmnEgHNlfUk5eWSGZKvN+lhA39pEifNGlIJs99eTaXnDiEO1/bxBULllB+QAPT8mGlFXUU6sK2dhQM0melJsbx84uncMelU1i9s4Zzf6c1l+TDSivqNFX1AxQM0uddNH0IT900i4zkOK5csJg/vlGqWUsCQFV9E9UNh7UUxgcoGCQqjB2QzjNfms05EwfwkxfWc9PDy6nThkBR78jieTpjaE/BIFEjLTGOu66Yznc+MY4Xi/fy6bveZMu+er/LEh+VKhg6pGCQqGJmfOGMkfx53kwq6w7x6bve1GJ8Uay0op6EuBgGZyf7XUpYUTBIVJo1Ko+nb5pNv/REPrfwHf68eJvfJYkPNlfUUZiXSmyMtvNsS8EgUasgN4UnbzyV00fn8f2nivnBs2to0TaiUaW0ol5TVTugYJColp4Uz4KrT+KaWcP53ze38qWHl2sToChxqLmF7VUNFOqK5w9RMEjUi40xbjtvAt/75Am8uGYvVy1YonWWosCWffW0tDpG91cwfJCCQSToutMKuXPudFbtquHSP76tJbz7uJKywIykMf3Tfa4k/CgYRNr45OSBPHDNDPbUNHLxPW+xVdNZ+6yNZbXExpjGGDqgYBD5gFNG5vLw9TOpP9TMxfe8zbo9B/wuSTxQUlbLsNwUEuNi/S4l7CgYRDoweUgWf7vhFOJijLn3LaZ4V43fJUkP21hWx5h+6kbqiIJBpBOj+qXz2BdOITUhjivuW8zKHfv9Lkl6SOPhFrZW1jNGA88dUjCIdKEgN4VHv3AymSnxXLVgCcu2VftdkvSAzRX1tDoYrYHnDikYRLoxJDuFx75wCrlpCXx+4Ts6c+gDSspqAc1I6kzIwWBms83smuDtfDMb0U37hWZWbmbFndx/gZmtMrMVZrbUzGYfXekivWdgZjKPzD+Z7NQEPvunJRpziHAlZbXExRgj8jQjqSMhBYOZ3QZ8C/hO8EvxwEPdPOx+4Jwu7n8FmOKcmwpcCywIpRYRvwzMTObh62eSnhTPZ/+0hPV7NVspUpWU1TE8L5WEOHWadCTU78qFwPlAPYBzbjfQ5TmYc24RUNXF/XXuP7ulpAJapEbC3pDsFB6+fiaJcbFcteAdtlXqOodItLG8VgPPXQg1GJqCv8QdgJn1yPmXmV1oZuuB5wmcNYiEvWG5qTx03QxaWlu56k9LKNNe0hHlYFNgjaTRmqraqVCD4TEz+yOQZWbXAy8D9x3vizvn/s85Nw74NPDDztqZ2fzgOMTSioqK431ZkeM2ql86918zg8q6Jj73p3eoaTjsd0kSotKKOpzTwHNXQgoG59wvgceBJ4CxwK3Oud/3VBHBbqeRZpbXyf33OueKnHNF+fn5PfWyIsdlytAs7v1sEVv21XPtA+9qVdYI8Z8ZSepK6kyog8+pwKvOuW8SOFNINrP443lhMxtlZha8PR1IACqP5zlFetvs0Xn85vKpLN9ezVf/+p72c4gAJWV1xMcawzUjqVOhdiUtAhLNbDCBbqRrCMw66pSZPQK8DYw1s51mNs/MbjCzG4JNPgMUm9kK4C7gsjaD0SIR49xJA/neJ8fzjzVl/PC5tei/cXjbWFbLiLxU4mM1I6kzcSG2M+dcg5nNA37vnPu5mb3X1QOcc3O7uf9nwM9CfH2RsDZv9gh27z/In/69hcFZyVx/eqHfJUknSsprmTwky+8ywlqokWlmdgpwJYEZRBB6qIhEhe+eewLnThrAj19Yxz/W7PW7HOlAQ1MzO6oOavG8boQaDF8Fvg086ZxbE7zq+VXvyhKJPDExxh2XTmXy4ExueXQFa3br6uhws6n8yOY8GnjuSqjB0AC0AnPNbBXwDPARz6oSiVBJ8bHc97kiMpLiuf6BpZTX6hqHcLJhb2BG0tgBOmPoSqjB8BdgIXARcB7wqeC/IvIB/TKSWHB1EdUNh5n/4DJNYw0jJWW1JMTFMCxXM5K6EmowVDjnnnXObXHObTvy4WllIhFs4uBMfn3ZFFbs2M8Pnl3jdzkSVFJWx6j8NGJjzO9SwlqoA8i3mdkCAgvfHTryRefck55UJdIHnDNxIDfOGcndr5cyeUgWc2cU+F1S1Cspq+Xkwly/ywh7oQbDNcA4Aquqtga/5gAFg0gXvn72WFbvquG2p9dwwsAMpg7VNEm/1Bw8zJ6aRi2FEYJQu5KmBJekuNo5d03wQ4veiXQjNsb43eXTyE9P5IsPLaOy7lD3DxJPbCo/MvCsGUndCTUYFpvZeE8rEemjslMT+ONnT6SyvombH11Bq5bN8MWGvYGpqlpVtXuhBsNsYIWZbQjuurY6OG1VREIwcXAmt503nn9t3Medr23yu5yoVFJWS2pCLIOzkv0uJeyFOsbQ1U5sIhKCK2YU8O6WKn79cgknDstm1qgOFxMWj2zYW8vo/unEaEZSt0JddntbRx9eFyfSl5gZP7pwEiPz0/jqX9+jXBv89KqN5bWM1cBzSLS8oEgvSk2M4w9XTqfuUDM3P7pCy3T3kn11h9hX18RoLYUREgWDSC8b3T+dH5w/gbdKK7nnjVK/y4kKRzbn0VIYoVEwiPjg0qKhnDdlEHe8VMKybVV+l9PnbSwLzEhSV1JoFAwiPgiMN0xkcFYyX3lkhfaM9tiGslqyUuLJT0/0u5SIoGAQ8UlGUjy/mzuNsgON/PdTq7Xzm4dK9tYypn86wd2EpRueBYOZLTSzcjMr7uT+K4PXRKwys7fMbIpXtYiEq6lDs7jlY2N4ftUenly+y+9y+iTnHBvKarUHw1Hw8ozhfrq+/mELcIZzbjLwQ+BeD2sRCVs3nDGSGSNyuPXpYrZV1vtdTp+z90AjtY3NGl84Cp4Fg3NuEdDpqJpz7i3nXHXw08XAEK9qEQlnsTHGry+bSkyMcfOjK2huae3+QRKytbsPADBuYIbPlUSOcBljmAe84HcRIn4ZnJXMjy+cxHvb93PXa5rC2pPW7D6AGYxXMITM92Aws48QCIZvddFmvpktNbOlFRUVvVecSC86b8ogPj11EL97dSMrd+z3u5w+o3hXDSPyUklNDHUFIPE1GMxsMrAAuMA5V9lZO+fcvcFlv4vy8/N7r0CRXvaDCybSLz2RWx5bwcEmbQnaE9bsPsDEQZl+lxFRfAsGMysgsNHPZ51zJX7VIRJOMpPj+eUlU9hcUc9PX1jndzkRr7q+iV37DzJxsLqRjoaX01UfAd4GxprZTjObZ2Y3mNkNwSa3ArnA3Wa2wsyWelWLSCSZNSqPa2YN54G3t7GoRF2nx2NNcOB5gs4YjopnnW7Oubnd3H8dcJ1Xry8Syb51zjgWlVTwrSdW8eLNp5OZHO93SRGpeHcNABMG6YzhaPg++CwiH5YUH8uvLp1Kee0hbn92rd/lRKziXTUMyU4mKyXB71IiioJBJExNHZrFjXNG8sTynby0tszvciKSBp6PjYJBJIx9+aOjOWFgBt95cjVV9U1+lxNRahsPs2VfvbqRjoGCQSSMJcTFcMelU6g52MStT3e47Jh0Yt2ewB4MEwfrjOFoKRhEwtwJAzP46pmjeW7VHp5ftcfvciJG8a7gwLOmqh41BYNIBLjhjJFMGpzJ958uZl/dIb/LiQjFu2vIT0+kX3qS36VEHAWDSASIi43hV5dOoa6xme8/Vay9G0KwdvcBJmp84ZgoGEQixJj+6dz8sdG8ULyXZ9Wl1KXGwy1sLK/T+MIxUjCIRJD5pxUyZWgWtz1dTEWtupQ6s27PAVpanWYkHSMFg0gEiYuN4ZcXT6b+UIu6lLqwIrg67ZShWT5XEpkUDCIRZnT/dG752BheXLOX59Sl1KH3tu9nYGYSAzOT/S4lIikYRCLQ9aeNYMrQLG7VLKUOLd9ezbQCnS0cKwWDSAR6v0upSV1KH1Re28jO6oNML8j2u5SIpWAQiVCj+6dzy1ljeKFYXUptvbc9ML6gM4Zjp2AQiWBtu5Q0Syngve37iY817cFwHBQMIhEsLjaGX12iLqW2lm+vZvygTJLiY/0uJWIpGEQi3Kh+gS6lF9fs5ZmVu/0ux1fNLa2s2rmfaZqmely83NpzoZmVm1mHS0Ka2Tgze9vMDpnZN7yqQyQaXH/aCKYOzeK2Z9ZQXtvodzm+Wb+3lsbDrUwfpoHn4+HlGcP9wDld3F8FfAX4pYc1iESFuNgYfnnJFA42tfDfT66O2i6l97ZXA+iM4Th5FgzOuUUEfvl3dn+5c+5d4LBXNYhEk1H90vjmx8fy8rpynly+y+9yfPHe9v3kpycyJFsXth0PjTGI9CHXzBpB0bBs/ufZNeypOeh3Ob1u+fZqpg3Nwsz8LiWiRUQwmNl8M1tqZksrKir8LkckbMXGGL+8ZArNLY5vPRFdXUpV9U1srWxgmi5sO24REQzOuXudc0XOuaL8/Hy/yxEJa8PzUvnOueNYVFLBI+/s8LucXvP++IIubDtuEREMInJ0rpo5jFmjcvl/z69le2WD3+X0iiVbqkiIjWGqBp6Pm5fTVR8B3gbGmtlOM5tnZjeY2Q3B+weY2U7ga8D3gm20eLpID4iJMX5+8RRizPjG4ytpbe37XUqLN1cytSBLF7b1gDivntg5N7eb+/cCQ7x6fZFoNzgrmVvPG89/Pb6KhW9u4brTCv0uyTMHGg9TvKuGL310tN+l9AnqShLpwy45cQhnjuvHz/+xgY1ltX6X45mlW6todXByYY7fpfQJCgaRPszM+MlnJpGWGMdX/7qCpuZWv0vyxOLNgfEFLbXdMxQMIn1cv/QkfnrRJNbuOcBvXi7xuxxPaHyhZykYRKLA2RMGcFnRUO55o5R3t3a6IEFEOjK+cHJhrt+l9BkKBpEo8f3zxjMkO4VbHl1BzcG+sxKNxhd6noJBJEqkJcbx28unsremke88uarPXBWt8YWep2AQiSLTCrL5xsfH8vfVe3n4ne1+l9MjNL7Q8xQMIlFm/mmFnD4mn9ufXcv6vQf8Lue4aHzBGwoGkSgTE2PccekUMpLjuekvy6ltjNzxhne3aHzBCwoGkSiUl5bIby+fytbKBr7+WOQumfGvjftIitf4Qk9TMIhEqVNH5vHf557AP9eWcedrm/wu55i8vqGcUwpzNb7QwxQMIlHs2lnDuXDaYH79cgkvry3zu5yjsnVfPVsrGzhjjJbi72kKBpEoZmb85KJJTBiUwc2PrmDVzv1+lxSyN0oCm3bNGdvP50r6HgWDSJRLio/lvs8VkZkcz9UL36EkQhbbe6OkguG5KQzPS/W7lD5HwSAiDMxM5uHrZxIfG8OVC5awdV+93yV1qfFwC2+V7lM3kkcUDCICwLDcVB66bibNLa3MvW8xLxbvDduro9/ZUkXj4VZ1I3lEwSAi7xvTP50/z5tJamIcNzy0jEv/+DbLtlWFXUC8UVJBQlyMLmzziJdbey40s3IzK+7kfjOz35nZJjNbZWbTvapFREI3cXAmL371NH504US27GvgM394m9N/8Rr/88wa3ty0LyyueXh9QzkzR+SQnKBpql7w8ozhfuCcLu7/BDA6+DEf+IOHtYjIUYiLjeHKmcN4/Ztz+NGFExndL51H3tnOlQuW8Knf/5uX1pb5dhaxo6qB0op6jS94yMs9nxeZ2fAumlwAPOgC/7sd/WIlAAAMoUlEQVQWm1mWmQ10zu3xqiYROTppiXFcOXMYV84cxsGmFv6+eg+/f3Uj1z+4lMlDMvnDVScyOCu5V2t6XdNUPefnGMNgYEebz3cGvyYiYSg5IZbPnDiEl792Br+4eDKbK+r5hg/Laby6royCnBRG5muaqlf8DAbr4Gsd/g8zs/lmttTMllZUVHhcloh0JS42hkuKhvK9T57A25srefDtrb322g1NzbxZWslZJ/THrKNfIdIT/AyGncDQNp8PAXZ31NA5d69zrsg5V5Sfr35FkXBw2UlDmTM2n5++uJ4tvXTdw7827qOpuZWzTlA3kpf8DIZngM8FZyedDNRofEEkcpgZP/vMZBJiY/j6Yyto6YUupZfXlpGeFMdJI7TMtpe8nK76CPA2MNbMdprZPDO7wcxuCDb5O7AZ2ATcB9zoVS0i4o3+GUncfsFElm/fz09fWOfpa7W2Ol7bUM6csf2Ij9UlWF7yclbS3G7ud8BNXr2+iPSOC6YOYvn2au771xZy0xK54YyRnrzOip372VfXpG6kXuBZMIhIdDAz/ue8CVTVN/HTF9aTk5LApScN7f6BR+nltWXExhhzxigYvKZgEJHjFtgudCo1Bw/z7SdXkZ2awMfG9+/R13hlXTkzhueQmRLfo88rH6aOOhHpEQlxMdxz1YlMGpLFlx5ezrJtVT323DuqGthQVsuZ6kbqFQoGEekxqYlxLLy6iEFZycx7YCmbyntmb4eX1wV2l+vpsxDpmIJBRHpUbloiD147g7iYGK5e+C57axqP+zlfWL2XMf3TGJarq517g4JBRHrc0JwU7r/mJPY3NDHvgXdpaGo+5ucqP9DIu9uq+OSkQT1YoXRFwSAinpg4OJM7r5jOuj0H+Nqjx76m0gvFe3EOPjl5QA9XKJ1RMIiIZz4yrh/f/eR4Xlyzl1/+c8MxPcfzq/Ywpn8ao/ql93B10hkFg4h46tpZw5k7Yyh3v17KE8t2HtVjj3QjnTtpoEfVSUcUDCLiKTPj9gsmckphLt95cjXLt1eH/Nj3u5EUDL1KwSAinouPjeHuK6czIDOJL/x5GXtqDob0uOdXB7qRRvdXN1JvUjCISK/ITk1gwdVFNBxqZv6DyzjY1NJl+/IDjby7Vd1IflAwiEivGdM/nd9ePo3i3TV88/GVXe4brW4k/ygYRKRXnTW+P//18XE8t2oPv3l5Y4dtnHM8vGQ7JwzMUDeSDxQMItLrbjijkItPHMJvX9nI0yt2fej+10sq2FBWy7zZI3yoThQMItLrzIwfXziJGSNy+Objq1i2rf1MpXvf2MyAjCTOn6Krnf2gYBARXyTExfDHq05kYGYS8x9cytbgvtGrd9bw9uZKrpk1nIQ4/Yryg6ffdTM7x8w2mNkmM/t2B/cPM7NXzGyVmb1uZkO8rEdEwkt2agL/+/mTaHWOz//vO1TWHeKPi0pJS4xj7swCv8uLWl7u+RwL3AV8AhgPzDWz8R9o9kvgQefcZOB24Cde1SMi4akwP40FV5/EnppGrvrTO/x99R6umFlARpI25PGLl2cMM4BNzrnNzrkm4K/ABR9oMx54JXj7tQ7uF5EocOKwbH43dxrr9x4gxoxrZg33u6So5uXWnoOBHW0+3wnM/ECblcBngN8CFwLpZpbrnKts28jM5gPzAQoKdHop0hd9fMIA7r5iOrWHmhmYmex3OVHNyzMG6+BrH7ya5RvAGWb2HnAGsAv40MLtzrl7nXNFzrmi/Pz8nq9URMLCJyYN5NKioX6XEfW8PGPYCbR9h4cAu9s2cM7tBi4CMLM04DPOuRoPaxIRkW54ecbwLjDazEaYWQJwOfBM2wZmlmdmR2r4DrDQw3pERCQEngWDc64Z+BLwD2Ad8Jhzbo2Z3W5m5webzQE2mFkJ0B/4kVf1iIhIaKyrRazCUVFRkVu6dKnfZYiIRBQzW+acKwqlrS4rFBGRdhQMIiLSjoJBRETaUTCIiEg7ETf4bGYVwLZeerk8YF8vvZaX+spxgI4lXPWVY+krxwEfPpZhzrmQrhCOuGDoTWa2NNRR/HDWV44DdCzhqq8cS185Dji+Y1FXkoiItKNgEBGRdhQMXbvX7wJ6SF85DtCxhKu+cix95TjgOI5FYwwiItKOzhhERKQdBUMbZvbD4P7TK8zsn2Y2qJN2V5vZxuDH1b1dZ3fM7Bdmtj54LP9nZlmdtNtqZquDxxuWC1AdxbF0ub94ODCzS8xsjZm1mlmns0Ui5H0J9VjC+n0xsxwzeyn4s/ySmWV30q4l+H6sMLNnOmrjl+6+x2aWaGaPBu9fYmbDu31S55w+gh9ARpvbXwHu6aBNDrA5+G928Ha237V/oMazgbjg7Z8BP+uk3VYgz+96j/dYgFigFCgEEgjsDDje79o7qPMEYCzwOlDURbtIeF+6PZZIeF+AnwPfDt7+dhc/K3V+13qs32PgxiO/ywhsf/Bod8+rM4Y2nHMH2nyayod3nAP4OPCSc67KOVcNvASc0xv1hco5908XWPYcYDGBTZIiUojHEsr+4r5zzq1zzm3wu46eEOKxRML7cgHwQPD2A8CnfazlWITyPW57jI8DZ5pZRztsvk/B8AFm9iMz2wFcCdzaQZOO9rIe3Bu1HaNrgRc6uc8B/zSzZcF9tcNdZ8cSae9JdyLtfelMJLwv/Z1zewCC//brpF2SmS01s8VmFk7hEcr3+P02wT+yaoDcrp7Uy609w5KZvQwM6OCu7zrnnnbOfRf4rpl9h8BGQ7d98Ck6eGyvT+3q7jiCbb5LYA/tv3TyNLOcc7vNrB/wkpmtd84t8qbizvXAsYTFewKhHUsIIuZ96e4pOvhaWP2sHMXTFATfk0LgVTNb7Zwr7ZkKj0so3+Ojfh+iLhicc2eF2PRh4Hk+HAw7Cew8d8QQAv2svaq74wgOin8KONMFOxc7eI7dwX/Lzez/CJyW9vovoB44lm73F+8tR/H/q6vniIj3JQRh8b50dRxmVmZmA51ze8xsIFDeyXMceU82m9nrwDQCfft+C+V7fKTNTjOLAzKBqq6eVF1JbZjZ6Dafng+s76DZP4CzzSw7OIPh7ODXwoaZnQN8CzjfOdfQSZtUM0s/cpvAcRT3XpWhCeVYCGF/8UgRKe9LiCLhfXkGODKz8GrgQ2dCwZ/1xODtPGAWsLbXKuxaKN/jtsd4MfBqZ38svs/vUfVw+gCeIPBDuAp4Fhgc/HoRsKBNu2uBTcGPa/yuu4Pj2ESgT3FF8OPIjIRBwN+DtwsJzGBYCawh0D3ge+3HcizBz88FSgj8FReux3Ihgb/eDgFlwD8i+H3p9lgi4X0h0Nf+CrAx+G9O8Ovv/8wDpwKrg+/JamCe33V/4Bg+9D0GbifwxxRAEvC34M/SO0Bhd8+pK59FRKQddSWJiEg7CgYREWlHwSAiIu0oGEREpB0Fg4iItKNgkKhhZnXH+fjHg1e+dtXm9a5WGw21zQfa55vZi6G2FzleCgaREJjZBCDWObe5t1/bOVcB7DGzWb392hKdFAwSdSzgF2ZWHNz34LLg12PM7O7gPgPPmdnfzezi4MOupM1VsWb2h+CiamvM7AedvE6dmf3KzJab2Stmlt/m7kvM7B0zKzGz04Lth5vZv4Ltl5vZqW3aPxWsQcRzCgaJRhcBU4EpwFnAL4Lr5FwEDAcmAdcBp7R5zCxgWZvPv+ucKwImA2eY2eQOXicVWO6cmw68Qft1t+KcczOAm9t8vRz4WLD9ZcDv2rRfCpx29IcqcvSibhE9EWA28IhzrgUoM7M3gJOCX/+bc64V2Gtmr7V5zECgos3nlwaXxI4L3jeewFIqbbUCjwZvPwQ82ea+I7eXEQgjgHjgTjObCrQAY9q0Lyew3ISI5xQMEo0626Skq81LDhJYcwYzGwF8AzjJOVdtZvcfua8bbdefORT8t4X//BzeQmDdoSkEzuYb27RPCtYg4jl1JUk0WgRcZmaxwX7/0wksLvZv4DPBsYb+tF9efR0wKng7A6gHaoLtPtHJ68QQWM0S4Irg83clE9gTPGP5LIFtG48YQ+SusioRRmcMEo3+j8D4wUoCf8X/l3Nur5k9AZxJ4BdwCbCEwG5XENibYw7wsnNupZm9R2D1083Am528Tj0wwcyWBZ/nsm7quht4wswuAV4LPv6IjwRrEPGcVlcVacPM0pxzdWaWS+AsYlYwNJIJ/LKeFRybCOW56pxzaT1U1yLgAhfYZ1zEUzpjEGnvOTPLAhKAHzrn9gI45w6a2W0E9s/d3psFBbu77lAoSG/RGYOIiLSjwWcREWlHwSAiIu0oGEREpB0Fg4iItKNgEBGRdhQMIiLSzv8HtoVCma5daLkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xbf60fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('alpha is', 0.15577457089129826)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>coef_lasso</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>columns</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.537611</td>\n",
       "      <td>0.635529</td>\n",
       "      <td>0.619655</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.184286</td>\n",
       "      <td>0.297763</td>\n",
       "      <td>0.280457</td>\n",
       "      <td>season</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.007874</td>\n",
       "      <td>0.008185</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>yr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>-0.001075</td>\n",
       "      <td>0.017236</td>\n",
       "      <td>mnth</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>-0.033981</td>\n",
       "      <td>-0.033886</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>-0.071831</td>\n",
       "      <td>-0.068944</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>-0.000000</td>\n",
       "      <td>-0.111401</td>\n",
       "      <td>-0.110003</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.116953</td>\n",
       "      <td>-0.214388</td>\n",
       "      <td>-0.211558</td>\n",
       "      <td>weathersit</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   coef_lasso   coef_lr  coef_ridge     columns\n",
       "6    0.537611  0.635529    0.619655       atemp\n",
       "0    0.184286  0.297763    0.280457      season\n",
       "4    0.000000  0.007874    0.008185  workingday\n",
       "1    0.000000  0.000000    0.000000          yr\n",
       "2    0.000000 -0.001075    0.017236        mnth\n",
       "3   -0.000000 -0.033981   -0.033886     holiday\n",
       "7   -0.000000 -0.071831   -0.068944         hum\n",
       "8   -0.000000 -0.111401   -0.110003   windspeed\n",
       "5   -0.116953 -0.214388   -0.211558  weathersit"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mses = np.mean(lasso.mse_path_,axis=1)\n",
    "plt.plot(np.log10(lasso.alphas_),mses)\n",
    "#plt.plot(np.log10(lasso.alphas_)*np.ones(3),[0.3,0.4,1.0])\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()\n",
    "\n",
    "print('alpha is',lasso.alpha_)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(columns),\"coef_lr\":list(lr.coef_.T),\"coef_ridge\":list(ridge.coef_.T),\"coef_lasso\":list(lasso.coef_.T)})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "完毕！"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
